document2md.cutter

Slices the Markdown converted from a section’s scanned page images down to just the section of interest, by locating its title and the next section’s title.

Slice the Markdown OCR’d from a note’s scanned page images down to just the note of interest.

The page images downloaded for a note (dofjson.download_nota_imagenes) hold whole pages: the first page usually begins with the tail of the previous note, and the last page often ends with the start of the next one. But the per-day note index gives us the title of every note in order, so we can locate two boundaries in the OCR’d text — where THIS note’s title appears (start) and where the NEXT note’s title appears (end) — and keep only what lies between.

Matching is fuzzy on purpose: OCR text differs from the index title in case, accents, line breaks and the odd misread character, so titles are compared on an accent-folded, marker-stripped, whitespace-collapsed form, and located with difflib’s longest-matching-block alignment rather than an exact search.

_normalize_with_map(text: str) → tuple[str, list[int]][source]

Return an accent-folded, lowercased, marker-stripped, single-spaced copy of text, alongside a map from each normalized-character position back to its index in the original text (so a match can be sliced from the original).

_locate(norm_text: str, idx_map: list[int], title: str, from_norm: int = 0) → tuple[int | None, int | None, float][source]

Best-effort match for title within norm_text[from_norm:]. Returns (normalized offset, original-text offset, 0..1 confidence), or (None, None, 0.0) if nothing matches.

from_norm lets the caller skip past an earlier region — used to find the NEXT note’s title after the current one, which matters when consecutive titles are near-duplicates (e.g. two deslinde avisos differing only in a number) and a whole-text search would otherwise anchor on the first.

Confidence is matched characters divided by the span of haystack they are spread across (floored at the title’s own length): a title that appears together, even split by a few OCR glitches, matches within a span close to its own length. Boilerplate that merely echoes some of the title’s words (e.g. a shared date phrase in running prose) racks up the same matched-character count but scattered over a much wider span, so it is penalized instead of being scored as if it were one solid match — that scattering is exactly what let such boilerplate be mistaken for a real title match before.

locate_titles(markdown: str, titulo: str, titulo_siguiente: str | None = None) → dict[source]

Locate the start (this note’s title) and end (next note’s title) boundaries in markdown. Exposed mainly for inspection/testing; cut_markdown_by_titles() is the usual entry point.

Returns a dict with start, start_confidence, end, end_confidence (offsets are into markdown; end is None when there is no next title or it wasn’t found after the start).

cut_markdown_by_titles(markdown: str, titulo: str, titulo_siguiente: str | None = None, min_confidence: float = 0.6) → str[source]

Return the slice of markdown that belongs to the note titled titulo.

titulo and titulo_siguiente are the note’s own title and the next note’s title (in publication order) from the per-day index — the next title marks where this note ends.

A boundary is only applied when its match confidence clears min_confidence: a start below the threshold falls back to the beginning of the text, and a weak/absent next-title match falls back to the end, so a poor match degrades to keeping more text rather than dropping the note’s content.

_trim_preceding_headings(markdown: str, end: int) → int[source]

Move end back over any run of blank and heading (#…) lines that immediately precede it, returning the new offset.