Show Bid Request Web Bot/Data Miner Bid Request Id: 30925 Bookmark in my 'To Do' list Posted by: Charlie10 (3 ratings) (Software buyer rating 10) Non-action Ratio: Very Good - 14.29% Buyer Security Verifications: Excellent Approved on: Oct 13, 2002 12:36:29 PM EDT Bidding Closes: Oct 17, 2002 12:40:58 PM EDT Viewed (by coders): 568 times Deadline: 10/18/2002 TIME EXPIRED Phase: 100% of work completed and accepted. Coder has been paid. Max Accepted Bid: Bidding is closed Project Type: Small Business Project: $100 (USD) + Bidding Type: Open Auction Categories: Web, Language Specific, C++ / C, Visual Basic, Web Services Enter chat room for this bid request (0 active users at Sep 30, 2003 6:19:46 AM EDT) Description: Web Bot/Data Miner: I require a client-side web bot/data miner to multi-thread out, retrieve data, and display select multiple part info from up to 12 different URL's. Concept & format is similar to display style as on http://www.pricewatch.com/ & http://shopper.cnet.com/ Speed of the application is critical. Total time to retrieve & display all data on broadband connection should be less than 5 seconds. C++ & winsock controls should probably be used for the development platform however I am open to alternate suggestions. Retrieval from some sites will require any combination of POST, HTTPS, redirects, secure logon, and cookie handling. Web connection methods should allow for dun & direct internet connection. DUN should spawn if no connection detected. Although not a current requirement, data parsing methods should be based to allow for future live-updating to accomodate changes to sites html. Also future versions of agent should allow for dynamic ad box. Data to be displayed in single form multi-column listview box. All columns must be ascen/descen sortable. Line click should spawn default browser for part details. Data should be dynamically displayed as retrieved and not pause between results. As mentioned above, total time required to return & display all data should be less than 5 seconds when operated on broadband connection. Display format should be similar to attached file. Deliverables: 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Installation package that will install the software (in ready-to-run condition) on the platform(s) specified in this bid request. 3) Complete ownership and distribution copyrights to all work purchased. Platform: All versions of Windows - 95 through XP Must be 100% finished and received by buyer on: Oct 18, 2002 EDT Deadline legal notes: All times are expressed in the time zone of the site EDT (UT - 5). If the buyer omitted a time, then the deadline is 11:59:59 PM EDT on the indicated date. Additional Files: This bid request includes IMPORTANT additional attached files. Please download and read fully before bidding. Remember that contacting the other party outside of the site (by email, phone, etc.) on all business projects < $500 (before the buyer's money is escrowed) is a violation of both the software buyer and seller agreements. We monitor all site activity for such violations and can instantly expel transgressers on the spot, so we thank you in advance for your cooperation. If you notice a violation please help out the site and report it. Thanks for your help.   Bidding/Comments: All monetary amounts on the site are in United States dollars. Rent a Coder is a closed auction, so coders can only see their own bids and comments. Buyers can view every posting made on their bid requests. See all rejected bids (and all comments) Name   Bid Amount    Date   Coder Rating   This bid was accepted by the buyer! atauro (11 ratings) in Mumbai, Maharashtra, Maharashtra India Bid id: 346,002   $350 (USD) Oct 14, 2002 6:23:10 AM EDT  10 (Excellent)     Hi, I've attached a demo copy of a program, that I had previously worked on, and is similar to the program you want. The program parses through web pages and saves the mined data to a file category.txt. After running the program, please click on Go>> to start the mining process. After a few seconds, you will see the completion status. On a 56 K Modem, my program (full version) went through 5200 (approx) webpages in 6 hours. All these pages were part of the yahoo web directory 'http://dir.yahoo.com/'. Please see the sample file 'business_and_economy.txt' to get an idea of the data mined by the program. The last few lines gives the statistics. What is the available bandwidth on your broadband connection. I am sure the program will fly on broadband. The attached program was only for batch processing and does not have any effective user interface. The realtime application that you want would have a great looking user-interface and grab 12 URLs within 5 seconds. Regards A Tauro Attached File           There are 82 private replies to/from atauro. Click here to view them.