jvoisin
ff2dfe977b
feat: remove the ref parameter from url
...
This is used by (at least) Ghost (https://forum.ghost.org/t/ref-parameter-being-added-to-links/38335 )
Examples:
- https://blog.exploits.club/exploits-club-weekly-newsletter-66-mitigations-galore-dirtycow-revisited-program-analysis-for-uafs-and-more/
- https://labs.watchtowr.com/is-the-sofistication-in-the-room-with-us-x-forwarded-for-and-ivanti-connect-secure-cve-2025-22457/
2025-05-06 19:59:55 -07:00
NoelNegash
81c7669945
feat(sanitized): allow Spotify iframes
2025-05-02 16:25:17 -07:00
Frédéric Guillot
d33e305af9
fix(api): hide_globally categories field should be a boolean
2025-04-21 19:43:25 -07:00
Frédéric Guillot
c87c93d85f
feat(config): add SCHEDULER_ROUND_ROBIN_MAX_INTERVAL option
...
Add option to cap maximum refresh interval when RSS TTL, Retry-After, Cache-Control, or Expires headers specify excessively high values.
2025-04-11 15:40:32 -07:00
Frédéric Guillot
ef22e95f8b
feat: implement proxy URL per feed
2025-04-06 21:05:19 -07:00
Frédéric Guillot
c45b51d1f8
feat: use Cache-Control max-age and Expires headers to calculate next check
2025-04-06 16:24:00 -07:00
Frédéric Guillot
0af1a6e121
refactor: avoid logging twice the feed errors in the background worker
2025-04-06 15:39:40 -07:00
Frédéric Guillot
535fd050b7
feat: add proxy rotation functionality
2025-04-06 14:59:00 -07:00
Frédéric Guillot
51560f191f
fix(subscription): add /rss/feed.xml to the list of known feed URLs
2025-03-28 16:59:06 -07:00
Frédéric Guillot
e342a4f143
fix: address minor issues detected by Go linters
2025-03-24 20:48:46 -07:00
Frédéric Guillot
315e72c412
fix(rewrite): remove obsolete rule for webtoons.com
2025-03-06 20:11:03 -08:00
jvoisin
f916373f55
fix: allow the <b> tag
2025-03-06 19:27:30 -08:00
jvoisin
5353211206
fix: allow the <u> tag in feeds
2025-03-06 19:26:26 -08:00
AiraNadih
ad02f21d04
refactor(rewrite): reorganize referer rules and remove obsolete mapping
2025-03-02 19:40:52 -08:00
Maytham Alsudany
f01ff067a5
fix(processor): add missing quotation marks to import comments
2025-02-24 16:34:26 -08:00
jvoisin
117d711d7d
feat(urlcleaner): add more Google Analytics parameters
2025-02-22 17:07:59 -08:00
jvoisin
4a77e937af
perf(sanitizer): remove two useless calls to strings.ReplaceAll
...
The [strings.Fields](https://pkg.go.dev/strings#Fields ) considers `'\t', '\n',
'\v', '\f', '\r', ' ', U+0085 (NEL), U+00A0 (NBSP).` as spaces, so no need to
remove them beforehand.
This is a continuation of f2f60a8f73
2025-02-18 19:42:39 -08:00
Frédéric Guillot
462ba8d7f7
feat(sanitizer): allow img tags with only a srcset and no src attribute
2025-02-15 18:03:36 -08:00
Frédéric Guillot
6eedf4111f
fix(scraper): avoid encoding issue if charset meta tag is after 1024 bytes
2025-02-15 17:05:14 -08:00
Frédéric Guillot
af1f966250
test(encoding): add unit tests for CharsetReader function
2025-02-15 15:40:07 -08:00
Frédéric Guillot
7f54b27079
fix(rss): handle item title with CDATA content correctly
...
Fix regression introduced in commit a3ce03cc
2025-02-15 14:51:27 -08:00
Frédéric Guillot
a3ce03cc9d
feat(rss): add workaround for RSS item title with HTML content
2025-02-14 21:21:49 -08:00
Frédéric Guillot
f2f60a8f73
feat(sanitizer): improve text truncation with better space handling
2025-02-06 21:21:49 -08:00
Frédéric Guillot
e777f12490
fix(sanitizer): correct HTML tag name from tfooter to tfoot
2025-02-06 21:16:29 -08:00
Julien Voisin
7eb1d15315
refactor(date): use an else-if instead of two if statements
2025-02-06 19:44:12 -08:00
Julien Voisin
b193bc212a
refactor(xml): improve the performances of NewXMLDecoder
...
- Invert a condition to make the code more readable
- Extract the encoding directly from the slice of bytes instead of converting
it to string first.
2025-01-30 19:37:06 -08:00
Julien Voisin
7275bc808a
feat(urlcleaner): add trackers to the blocklist
2025-01-29 19:32:19 -08:00
Frédéric Guillot
369054b02d
feat(processor): fetch YouTube watch time in bulk using the API
2025-01-24 15:16:23 -08:00
Frédéric Guillot
c3c42b0c37
fix(scraper): update TechCrunch scraper rule
2025-01-23 19:29:32 -08:00
jvoisin
2e57e3351b
Remove superfluous parenthesis
2025-01-23 19:20:13 -08:00
jvoisin
a412cde3b3
Don't define receivers on both values and pointer
...
And use `o` instead of `outline` as done everywhere else.
2025-01-23 19:20:13 -08:00
jvoisin
abfd9306a4
Guard against a potential null dereference
2025-01-23 19:20:13 -08:00
Frédéric Guillot
1faccc7eca
fix(sanitizer): non-allowed attributes are not properly stripped
...
Regression introduced in commit 58178d90cb
2025-01-22 20:50:38 -08:00
Frédéric Guillot
9c82e55b98
fix: do not strip tags in Atom entry title
2025-01-18 15:33:44 -08:00
Frédéric Guillot
e9520f5d1c
fix(finder): do not add redirections to the list of subscriptions to avoid confusion
2025-01-12 17:09:32 -08:00
Jake Walker
6cbe8c3a9d
feat: add fix_ghost_cards rewrite rule
2025-01-12 14:43:27 -08:00
Julien Voisin
f116f7dd6a
test(sanitizer): add a fuzzer
2025-01-11 17:19:31 -08:00
Frédéric Guillot
5549f75dd7
fix(sanitizer): allow <hr> tags
2024-12-27 13:56:06 -08:00
Julien Voisin
8df4b780a8
refactor(readingtime): replace whatlanggo with an ad-hoc implementation
...
The package `github.com/abadojack/whatlanggo` is unmaintained since 5 years, is
overkill for simply detecting CJK, and is quite slow.
2024-12-26 14:21:07 -08:00
Julien Voisin
195b75d185
refactor(rewriter): use custom title case converter implementation instead of golang.org/x/text/cases.Title()
...
The implementation is equivalent to
`cases.Title(language.English).String(strings.ToLower(…))`,
and this is the only place in miniflux where
"golang.org/x/text/cases" and "golang.org/x/text/language"
are (directly) used.
This reduces the binary size from 27015590 to
26686112 on my machine.
Kudos to https://gsa.zxilly.dev for making it straightforward to catch things
like this.
2024-12-23 21:16:02 -08:00
jvoisin
bd91e5f320
Add more referer spoofing
...
Based on #2261 . For moyu.im/jandan.net, see https://github.com/DIYgod/RSSHub/issues/11528
2024-12-20 11:53:38 -08:00
Sevi.C
bca9bea676
feat: add date-based entry filtering rules
2024-12-16 20:38:20 -08:00
jvoisin
7939b54341
Resize favicons to 32x32 to account of scaling
...
As suggested by @michaelkuhn in https://github.com/miniflux/v2/pull/2998#issuecomment-2546702212
2024-12-16 19:28:38 -08:00
jvoisin
2df59b4865
Refactor internal/reader/readability/testdata
...
- Use chained strings.Contains instead of a regex for
blacklistCandidatesRegexp, as this is a bit faster
- Simplify a Find.Each.Remove to Find.Remove
- Don't concatenate id and class for removeUnlikelyCandidates, as it makes no
sense to match on overlaps. It might also marginally improve performances, as
regex now have to run on two strings separately, instead of both.
- Add a small benchmark
2024-12-15 20:52:32 -08:00
Julien Voisin
777d0dd248
feat: resize favicons before storing them
...
Some websites are using images of O(10kB) when not )O(100kB) for their
favicons. As miniflux only displays them with a 16x16 resolution, let's do our
best to resize them before storing them in the database. This should make
miniflux consume less bandwidth when serving pages, for the joy of mobile users
on a small data plan.
Of course, images that already are 16x16 aren't resized.
2024-12-15 20:47:19 -08:00
Julien Voisin
cfda948c3a
refactor(rewriter): avoid the use of regex in addDynamicImage
...
See https://dustri.org/b/parsing-noscript-tags-with-goquery.html for the whole
story.
2024-12-15 17:56:39 -08:00
Julien Voisin
945d436055
refactor(rewriter): replace regex with URL parsing for referrer override
...
No need for brittle regex when matching plain strings or domain names.
This should save some negligible amount of heap memory as well as
tremendously speeding up the matching.
2024-12-13 14:50:12 -08:00
Frédéric Guillot
c3649bd6b1
refactor(rewrite): remove unused function arguments
2024-12-12 21:10:35 -08:00
Julien Voisin
6ad5ad0bb2
refactor(readability): various improvements and optimizations
...
- Replace a completely overkill regex
- Use `.Remove()` instead of a hand-rolled loop
- Use a strings.Builder instead of a bytes.NewBufferString
- Replace a call to Fprintf with string concatenation, as the latter are much
faster
- Remove a superfluous cast
- Delay some computations
- Add some tests
2024-12-12 20:41:56 -08:00
Frédéric Guillot
113abeea59
test(rewrite): add unit test for referer rewrite function
2024-12-12 20:11:47 -08:00
