Chaining, retries, and general scraping durability

Browse Source

Make the scraping infrastructure more durable since the scraping service often fails to deliver

...

This commit is contained in:

NMerz

2020-11-01 11:30:03 -05:00

parent 79598bf9e9

commit ee04a06e0e

21 changed files with 74 additions and 19 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

2

Lambdas/Scraping/prefix_list_part2.txt

File diff suppressed because one or more lines are too long