mirror of
https://github.com/instructkr/claw-code.git
synced 2026-10-30 00:00:22 +08:00
Bellmanandclawdbot
fd8527f899
* Fix issue #3364: web search returns error on non-2xx response and filters search engine self-links - Add HTTP status code check to execute_web_search: return error if response is not 2xx instead of proceeding with potentially junk results (e.g., DuckDuckGo challenge pages) - Filter out links pointing back to the search engine host in extract_search_hits_from_generic_links to avoid returning search engine self-links when no proper search results are found - Add regression tests for both scenarios Refs #3364 * fix: implement registrable domain filtering and error on 202/blocked responses - Replace exact host matching with registrable domain comparison to properly filter search engine self-links across subdomains (e.g., html.duckduckgo.com vs duckduckgo.com) - Add error handling for 2xx responses with no search results (e.g., 202 challenge pages from blocked search providers) - Add regression tests for both 202 challenge page and mixed self-link scenarios - Remove unrelated Cargo.lock version bumps (revert to 0.1.6) Fixes #3364 * fix(tools): reject DuckDuckGo 202 challenge pages and drop search-engine links The challenge page arrives as HTTP 202, which is_success() accepts, and its links point at duckduckgo.com while the request goes to html.duckduckgo.com, so the previous filter kept them. Only a 200 page is parsed now, and the generic fallback drops any duckduckgo.com link or link to the configured search host. Removes the registrable-domain heuristic and the 'no hits is an error' path (a real query can legitimately return nothing), and replaces tests that never exercised a self-link with ones that fail on the old code. Refs #3364 --------- Co-authored-by: clawdbot <clawdbot@ultraworkers.dev>