Case 4.5 Spiders at the Auction : Case 4.5 Spiders at the Auction Wilson Waddell
Case 4.5 : Case 4.5 Case 4.5 deals specifically with Ebay and the AuctionWatch web spider but has broader implications in web ethics.
AuctionWatch is an auction price aggregator service that provides consumers with information on the cheapest price for a given good.
It uses information from sites such as Ebay and Amazon.com
Ebay has had several disputes with auction aggregator services.
What is an Auction Aggregator Service? : What is an Auction Aggregator Service? An auction aggregator service is a service that indexes the prices of auctions or items and then compares them on a single web site or database.
Typically these services go about gathering price information with the help of a web spider.
Web Spiders: What are they? : Web Spiders: What are they? A web spider is a program that browses the web and indexes/collects data.
Search engines such as Google use spiders to collect data to display for search queries.
The spider operates by visiting a set of seed URLs and then finds hyperlinks from those and adds them to its list of URLs to visit. Each URL is then revisited according to that programs set of revisiting policies.
Along the way the spider collects data according to its programming.
Spiders can perform these operations extremely rapidly.
The Issue : The Issue Ebay is a site that relies on its auctions and Paypal system in order to make a profit.
If a service allows indexing of auctions and compares them to other prices on the internet Ebay may lose money.
Additionally web spiders have an impact on Ebay’s server infrastructure.
Spiders’ Downsides : Spiders’ Downsides One issue Ebay took to having spiders crawling its site was server load.
Web spiders take considerable bandwidth to operate due to their high level of parallelism and speed. As such server load is increased and consumers may notice slowdowns.
Poorly written web spiders can crash servers.
Multiple web spiders operating on a the same server can cause slowdowns and server crashes.
Ethical Issues With Web Spiders : Ethical Issues With Web Spiders There are two main ethical concerns in relation to Case 4.5.
1. eBay claims that its auction data is protected-spiders used to compared prices may show high prices on eBay and lead consumers to choose other options for buying items.
2. Server strain is a real concern. In this case eBay has a valid right to say that they don’t want spiders used on their servers so that the customer experience will be pleasant as much as possible.
My Take on the Issue : My Take on the Issue Personally I don’t have a problem with web spiders being used in polite ways.
Creators of spider programs can program them to slow down their search speed. In the case of the AuctionWatch spider you could simply slow it down to a few seconds in order to lower server strain from the bot.
The server issues are valid and I don’t condone web spiders that cause significant server strain. This does not help the consumer or user group.
Some things to think on : Some things to think on Some things to think about are:
The inherent method of maintaining usability on the web is the web spider. Without it search engines would be harder to create and we might not have Google.
Most spiders have inbuilt time lapses for their page searches to not overload servers and cause slowdowns.
Is this technology malicious?