Skip to content

feat: add 30 Amazon, Walmart and flight entities to SerpApi questions (closes #342) - #347

Merged
houtanb merged 1 commit into
forecastingresearch:mainfrom
karoliskascenas-fri:feat/serpapi-more-questions
Nov 6, 2026
Merged

houtanb merged 1 commit into
forecastingresearch:mainfrom
karoliskascenas-fri:feat/serpapi-more-questions

Conversation

@karoliskascenas-fri

Copy link
Copy Markdown
Contributor

Adds 30 Amazon products, 30 Walmart groceries and 30 long-haul flights to the SerpApi questions
(#342). Data only; all changes are in src/sources/serpapi_questions.py.

  • Amazon: categories not yet covered, no food or drink. Each is a New offer sold by Amazon.com.
  • Walmart: national-brand groceries at 10 Supercenters in new states, three per store. Each is
    sold by Walmart.com and in stock at its store; store names were checked on Walmart's pages.
  • Flights: regions not yet covered. Each flies daily on the same route and is bookable nonstop
    under the same number on sample winter and summer 2027 dates.

Testing: local fetches returned valid measurements for all 90. Both SerpApi jobs were deployed
and run in forecastbench-karolis (240 resolution files). make lint and make test pass.

Cost: daily SerpApi searches go from 150 to 240, about 7,200 credits a month.

🤖 Generated with Claude Code

@houtanb houtanb assigned pythoryn and unassigned pythoryn Oct 26, 2026
@houtanb
houtanb requested a review from pythoryn October 26, 2026 07:54

@pythoryn pythoryn left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The additions generally look consistent with the existing source rules. I tested all 90 new entries in serpapi and through the PR’s parsers: 29/30 Amazon, 30/30 Walmart, and 28/30 flights returned valid measurements.

Three entries failed even with fresh retries:

  • AirTag (B0933BVK6T): Only third-party seller 6ave offers appeared; no qualifying new Amazon.com offer.
  • AR1132 (EZE–MAD): Google returned EZE–GIG / GIG–MAD legs, which do not match the configured airport pair.
  • BA7 (LHR–HND): The configured query returned no structured flight-status result, including with caching disabled.

Could these entries be investigated manually or replaced?

The claim that all added flights operate daily also needs checking. Published schedules show QF93 excluding Saturdays during November 15–23, 2026, and Thursdays during November 25–December 2; LH716 excludes Tuesdays during January 13–24, 2027, and Mondays during January 26–February 7.

Minor wording nit: the Burt’s Bees ASIN is a four-pack; adding that quantity would make its description consistent with the other products.

Comment thread src/sources/serpapi_questions.py Outdated
"asin": "B0000683A4",
},
{
"id": "b0933bvk6t-amazon-com-new",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only third-party seller offers appeared; no qualifying new Amazon.com offer.

Comment thread src/sources/serpapi_questions.py Outdated
"destination_name": "London Heathrow Airport",
},
{
"id": "ar1132-eze-mad",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Google returned EZE–GIG / GIG–MAD legs, which do not match the configured airport pair

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The configured query returned no structured flight-status in serpapi results

Comment thread src/sources/serpapi_questions.py Outdated
"destination_name": "Dallas Fort Worth International Airport",
},
{
"id": "qf93-mel-lax",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no flights Saturdays during November 15–23, 2026, and Thursdays during November 25–December 2

Comment thread src/sources/serpapi_questions.py Outdated
"destination_name": "Tokyo Haneda Airport",
},
{
"id": "lh716-fra-hnd",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no flights Tuesdays during January 13–24, 2027, and Mondays during January 26–February 7

@karoliskascenas-fri
karoliskascenas-fri force-pushed the feat/serpapi-more-questions branch from 2b449ec to 42bd531 Compare October 28, 2026 14:04
@karoliskascenas-fri

Copy link
Copy Markdown
Contributor Author

Thanks for testing all 90, and sorry for only getting back to this now! Updated and rebased on main:

  • AirTag: when I added it on Oct 23, Amazon.com was the pinned offer ($29.00) and the parser returned a value. Amazon.com has since dropped it, so I've replaced it with a GE surge protector 2 pack (B082JHB9PP).
  • AR1132, QF93, LH716: confirmed and replaced. A full check of all 30 timetables on flight.info turned up four more flights with gaps (AV10, BA55, MS985, QF7), so I've replaced those too.
  • BA7: I couldn't reproduce this. The status card came back for me, and the timetable has no gaps, so I've kept it.
  • Burt's Bees: renamed to "4 Pack".

All replacements return valid measurements through the parsers.

@karoliskascenas-fri
karoliskascenas-fri force-pushed the feat/serpapi-more-questions branch from 42bd531 to 5811817 Compare October 29, 2026 14:25
@pythoryn

Copy link
Copy Markdown
Collaborator

All issues have been addressed. I tried again BA7: this time worked fine. All good!

@karoliskascenas-fri
karoliskascenas-fri force-pushed the feat/serpapi-more-questions branch 2 times, most recently from 3f3b925 to ed39f2b Compare November 2, 2026 20:29
@houtanb
houtanb force-pushed the feat/serpapi-more-questions branch from ed39f2b to 7ca4f55 Compare November 6, 2026 16:05
@houtanb
houtanb force-pushed the feat/serpapi-more-questions branch from 7ca4f55 to b674d94 Compare November 6, 2026 21:06
@houtanb
houtanb merged commit 3db2f59 into forecastingresearch:main Nov 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants