Yahoo Search Script Fixed – Looking Backward At My Linux And Unix Mistakes

Written By
Web Webster
Web Webster
Jan 5, 2009

[ Thanks to Mike
Golvach
for this link. ]

“It turns out, the main SNAFU (situation normal, all
f***ed up) was that I didn’t include a –user-agent option to my
wget command… ooops.

“Check out the test run pictures below, to see what a difference
it makes, and feel free to laugh “with” or “at” me for the
bone-headed mistake. I’m immune to feeling any worse right now
😉

“For the newer pictures, I added a line to the “searchterms”
file to make sure I could get a result from the last search after
the few preceding it failed. Also, for this test, like the last, I
modified the script slightly to only go after the first 200
results, rather than the default maximum of 1000 that Yahoo will
return per search. As the newer pictures demonstrate, you don’t
need to go after the full 1000 like we did last time in order to
get bounced.”


Complete Story

Web Webster

Web Webster

Web Webster has more than 20 years of writing and editorial experience in the tech sector. He’s written and edited news, demand generation, user-focused, and thought leadership content for business software solutions, consumer tech, and Linux Today, he edits and writes for a portfolio of tech industry news and analysis websites including webopedia.com, and DatabaseJournal.com.

Linux Today Logo

LinuxToday is a trusted, contributor-driven news resource supporting all types of Linux users. Our thriving international community engages with us through social media and frequent content contributions aimed at solving problems ranging from personal computing to enterprise-level IT operations. LinuxToday serves as a home for a community that struggles to find comparable information elsewhere on the web.

Property of TechnologyAdvice. © 2026 TechnologyAdvice. All Rights Reserved

Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.