- Move the seeking inside of DetectFeedFormat instead of having it everywhere
in ParseFeed
- Provide a hint for the compiler to eliminate a useless bound check in
DetectJSONFormat, otherwise it'll check that buffer[i] is valid on every
iteration of the loop. This shouldn't make a big difference, but oh well.
- Use an iterator instead of generating a whole slice when iterating on the selection.
- Using an iterator allows to use a for-loop construct, instead of a lambda,
which is a bit clearer
- Do the filtering Find()'s selector, instead of in the loop, which doesn't
matter much now that we're using an iterator, but it makes the code a bit
more obvious/simpler, and likely reduces a bit the number of iterations.
There is no need to materialize the whole text content of the selection only to
count its number of commas. As we already have a getLengthOfTextContent
function that is pretty similar, this commit refactors it to make it more
generic, in the form of a map/fold(+).
- Don't define the queries before possible early returns
- Check for the presence of the href attribute in the queries, instead of later
on iterating on the selection
- Add two edge-cases to the tests
- Use EachIter instead of Each, if only to avoid the lambda
There is no need to try to match the regexp over the whole input, having it
anchored is enough. If we feel extra-lenient, we might strip spaces in
front/tail, but I don't think it's necessary.
This commit also invert a condition to reduce the level of nested indentation,
and make a condition stricter.
Instead of doing some ciphers manipulation before instantiating the http.Transport
and then assigning them, instantiate http.Transport, and then in an if do the
manipulation. This makes the code a bit clearer, which is always nice when it
comes to cryptographic shenanigans.
There is no need to allocate half a kilobyte of memory only check that a buffer
starts with a bunch of spaces and a `{`, 32b should be more than enough. Also,
no need to allocate it on the heap, having it on the stack works perfectly.
Instead of using an ugly (and incomplete) regex, let's use a simple for-loop to
parse ISO8601 dates, and make it explicit that we're only supporting a subset
of the spec, as we only care about youtube video durations.
- Grow the underlying buffer of SanitizeHTML's strings.Builder to 3/4 of the
raw HTML from the start, to reduce the amount of iterative allocations. This
number is a complete guesstimation, but it sounds reasonable to me.
- Add a `absoluteURLParsedBase` function to avoid parsing baseURL over and over.
When we're only interested in the length of contained Text, there is no need to
materialize it fully to then call len() on the result: we can simply iterate
over the text element and sum their length instead.
- There is no need to materialize all the content of a given Node when we can
simply compute its length directly, saving a lot of memory, on the order of
several megabytes on my instance, with peaks at a couple of dozen.
- One might object to the usage of a recursive construct, but this is a direct
port of goquery's Text method, so this change doesn't make anything worse.
- The computation of linkLength can be similarly computed, but this can go in
another commit, as it's a bit trickier, since we need to get the length of
every Node that has a `a` Node as parent, without iterating on the whole
parent chain every time.
Before
```console
$ go test -bench=.
goos: linux
goarch: arm64
pkg: miniflux.app/v2/internal/reader/readability
BenchmarkExtractContent-8 34 86102474 ns/op
BenchmarkGetWeight-8 10573 103045 ns/op
PASS
ok miniflux.app/v2/internal/reader/readability 5.409s
```
After
```console
$ go test -bench=.
goos: linux
goarch: arm64
pkg: miniflux.app/v2/internal/reader/readability
BenchmarkExtractContent-8 56 83130924 ns/op
BenchmarkGetWeight-8 246541 5241 ns/op
PASS
ok miniflux.app/v2/internal/reader/readability 6.026s
```
This should make ProcessFeedEntries marginally faster, while saving
some memory.
- Use an array of strings instead of a regex, like done in ef13756b1a7a7ba30fd34174a5367381fd8b4849
- Extract the `shouldRemove` function from `removeUnlikelyCandidates`, as there
is no reason to have it there instead of being a proper standalone function.
- Improve a condition, where the goquery selection would have its `id`
attribute left unchecked if a `class` one was present, regardless of if
`class` was a candidate to removal or not.
- Add some comments
