Archived

🐛 Fix random translation failures and stuck target language (closes #2) #8

Closed

jiaofangkai926 wants to merge 1 commit from jiaofangkai926/deepl-scraper:fix/issue-2-reliable-translation into master

jiaofangkai926 commented

2026-10-20 16:54:40 +02:00

First-time contributor

Problem

Issue #2 reports random translation errors, e.g.

No node found for selector: .lmt__source_textarea
    at translate (index.js:55)

when running something like translate("Mittel", "de-de", "ja-JA").

Root cause

The translation flow is a sequence of UI automation steps, but each step assumed the UI was already in the expected state:

The source textarea node may not be inserted yet when page.type is called → No node found for selector: .lmt__source_textarea, i.e. the failure reported in the issue.
Language option clicks can be silently ignored while the language panels are still swapping content → DeepL keeps the previously selected language (English) while the code believes the target is Japanese → wrong / random output, which explains the japanese-specific failures and inconsistent results.
Any transient selector/timeout error aborted the whole call immediately; there was no retry on a fresh page.

(The f684f108 commit already added a waitForSelector for the source textarea; this PR covers the remaining cases.)

Solution

chooseLanguage() now waits for the option's own panel and verifies the panel's label afterwards; if the expected language did not get selected, it re-opens the menu and retries (up to 3 attempts), failing with LANGUAGE_SWITCH_FAILED:source / LANGUAGE_SWITCH_FAILED:target if the switch really could not happen.
The source textarea is awaited with waitForSelector(..., { visible: true }) before typing.
The requestfinished listener is registered before typing, so a request fired right after the last keystroke cannot be missed.
translate() retries once on a fresh page when the failure looks transient (missing node / navigation or wait timeout); non-transient errors (e.g. TRANSLATION_RESULT_UNAVAILABLE, invalid language arguments) fail immediately instead of retrying forever.
Version bumped to 1.0.12, changelog updated.

Verification

node --check index.js passes.

I verified the repaired behavior deterministically with a local fixture server (fake DeepL page + translation JSON endpoint, driven by the real translate() via DEEPL_TRANSLATOR_HOMEPAGE, Chrome via PUPPETEER_EXECUTABLE_PATH). The fixture injects exactly the failure modes from this issue:

Scenario	Injected failure	Result
baseline	none	5/5 pass, `target.lang === 'ja'`
delayed textarea	source textarea inserted 1.5 s late	5/5 pass, old code threw
ignored target click	first click on the Japanese option silently ignored	5/5 pass after re-opening the menu, label verified
broken first load	first homepage lacks the Node, second load is complete	5/5 pass; fixture access log shows the automatic retry (`GET /translator` ×2)
empty result	translation endpoint returns no `result`	fails fast with `TRANSLATION_RESULT_UNAVAILABLE`, no retry
invalid arguments	e.g. `de-de` / `JA-ja`	`INVALID_*_LANGUAGE`, unchanged behavior

Note: DeepL's live site has since been redesigned (the old markup/selectors this module targets is from the 2019–2020 era), so live end-to-end verification against deepl.com is not possible with the original selectors; the fixture-based approach exercises the repaired logic itself.

Impact scope

Only index.js flow control changed; public API, return shape and error contracts are unchanged, two errors added and documented.
Browser pages are closed on every attempt (same as 1.0.4's intent), and the shared browser is reused.
No new dependencies.

Closes #2.

## Problem Issue #2 reports random translation errors, e.g. ``` No node found for selector: .lmt__source_textarea at translate (index.js:55) ``` when running something like `translate("Mittel", "de-de", "ja-JA")`. ## Root cause The translation flow is a sequence of UI automation steps, but each step assumed the UI was already in the expected state: 1. The source textarea node may not be inserted yet when `page.type` is called → **`No node found for selector: .lmt__source_textarea`**, i.e. the failure reported in the issue. 2. Language option clicks can be silently ignored while the language panels are still swapping content → DeepL keeps the previously selected language (English) while the code believes the target is Japanese → wrong / random output, which explains the *japanese-specific* failures and inconsistent results. 3. Any transient selector/timeout error aborted the whole call immediately; there was no retry on a fresh page. (The `f684f108` commit already added a `waitForSelector` for the source textarea; this PR covers the remaining cases.) ## Solution - `chooseLanguage()` now waits for the option's own panel and verifies the panel's label afterwards; if the expected language did not get selected, it re-opens the menu and retries (up to 3 attempts), failing with `LANGUAGE_SWITCH_FAILED:source` / `LANGUAGE_SWITCH_FAILED:target` if the switch really could not happen. - The source textarea is awaited with `waitForSelector(..., { visible: true })` before typing. - The `requestfinished` listener is registered **before** typing, so a request fired right after the last keystroke cannot be missed. - `translate()` retries **once** on a fresh page when the failure looks transient (missing node / navigation or wait timeout); non-transient errors (e.g. `TRANSLATION_RESULT_UNAVAILABLE`, invalid language arguments) fail immediately instead of retrying forever. - Version bumped to `1.0.12`, changelog updated. ## Verification `node --check index.js` passes. I verified the repaired behavior deterministically with a local fixture server (fake DeepL page + translation JSON endpoint, driven by the real `translate()` via `DEEPL_TRANSLATOR_HOMEPAGE`, Chrome via `PUPPETEER_EXECUTABLE_PATH`). The fixture injects exactly the failure modes from this issue: | Scenario | Injected failure | Result | | --- | --- | --- | | baseline | none | 5/5 pass, `target.lang === 'ja'` | | delayed textarea | source textarea inserted 1.5 s late | 5/5 pass, old code threw | | ignored target click | first click on the Japanese option silently ignored | 5/5 pass after re-opening the menu, label verified | | broken first load | first homepage lacks the Node, second load is complete | 5/5 pass; fixture access log shows the automatic retry (`GET /translator` ×2) | | empty result | translation endpoint returns no `result` | fails fast with `TRANSLATION_RESULT_UNAVAILABLE`, no retry | | invalid arguments | e.g. `de-de` / `JA-ja` | `INVALID_*_LANGUAGE`, unchanged behavior | Note: DeepL's live site has since been redesigned (the old markup/selectors this module targets is from the 2019–2020 era), so live end-to-end verification against `deepl.com` is not possible with the original selectors; the fixture-based approach exercises the repaired logic itself. ## Impact scope - Only `index.js` flow control changed; public API, return shape and error contracts are unchanged, two errors added and documented. - Browser pages are closed on every attempt (same as 1.0.4's intent), and the shared browser is reused. - No new dependencies. Closes #2.

jiaofangkai926 added 1 commit 2026-10-20 16:54:41 +02:00

🐛 Retry transient translation failures and verify language switching (fixes #2 ) 6e9306fedf

Random failures in the issue report were transient UI automation races:

* wait for the source textarea to be visible before typing
* verify the language panel really switched and re-open it when a click is ignored
* retry once on a fresh page for transient selector/timeout errors
* fail fast with LANGUAGE_SWITCH_FAILED / TRANSLATION_RESULT_UNAVAILABLE

KaKi87 commented

2026-10-20 17:12:34 +02:00

Owner

Hi,

Thank you for taking the time to submit a PR.

Actually, I've abandoned the development of deepl-scraper and several other scraping projects a while ago as I lost interest in playing the cat & mouse game with corporate developers.

Moreover, in the AI era, having a small model running locally (e.g. using MNN) will use less compute than running a headless Chrome browser.

I'm going to close this and archive this repository.

Sorry and good luck

Hi, Thank you for taking the time to submit a PR. Actually, I've abandoned the development of `deepl-scraper` and several other scraping projects a while ago as I lost interest in playing the cat & mouse game with corporate developers. Moreover, in the AI era, having a small model running locally (e.g. using [MNN](https://github.com/alibaba/MNN/tree/master/apps/mnncli)) will use less compute than running a headless Chrome browser. I'm going to close this and archive this repository. Sorry and good luck

KaKi87 closed this pull request

2026-10-20 17:12:34 +02:00

KaKi87 commented

2026-10-20 17:14:00 +02:00

Owner

PS: feel free to join my Discord community if you wish to continue this conversation though, cause I know the archiving action will make this read-only and I don't mean to prevent you from responding.

PS: feel free to [join my Discord community](https://discord.com/invite/7xjuuAnwbk) if you wish to continue this conversation though, cause I know the archiving action will make this read-only and I don't mean to prevent you from responding.

This repository is archived. You cannot comment on pull requests.

No reviewers