Commit Graph
5 Commits
Author SHA1 Message Date
upstream 03e823a030 Upstream adbar/trafilatura state of the touched files before 1377ec86cc 2026-10-02 17:03:46 +02:00
dualfroz 1884a449e8 Use the listing container only when it widens a short first entry
On correctiv.org the article page preloads the next twelve full stories as
sibling <article> elements, so with favor_recall the listing container took
all of them (98k chars instead of 9.6k). A teaser strip beside a short body
(mopo.de) was also taken over the body.

The container is now used only when the element BODY_XPATH picks on its own
is one of its entries and shorter than the recall sure-thing length
(10x min_extracted_size).
2026-10-10 15:26:09 +02:00
dualfroz ebc0ed3341 Use the listing expression only when its container holds most of the page text 2026-10-10 15:26:09 +02:00
dualfroz ce36d8e544 fix: keep every entry heading on listing pages (#774)
A blog index, archive or category page holds several <article> siblings.
BODY_XPATH's "(.//article)[1]" keeps the first entry only, so the page
looks under-extracted and escalates to justext, which emits paragraphs
and drops every heading.

Give the recall pass a listing expression that takes the container of
the sibling articles, so the retry returns all entries with their heads
and wins the escalation. Recall only: on an article page the same shape
is a related-posts strip, whose container would hide the body.
2026-10-10 15:26:09 +02:00
upstream 7568e2a786 Upstream adbar/trafilatura state of the touched files before 49484caaab 2026-09-11 23:09:08 +02:00