Extracting data from www sites

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
Pre sales tech question

Extracting data from www sites

Post by Pre sales tech question » Mon Oct 03, 2005 2:12 pm

Hi, was woundering if some can assist us.

We have a requirement and would like to know if this application can help us.

Heres what we would like :

Every morning at 4am we would like some sort of program to visit a number of web site and get the figures for us, we would like them exported into a Excel sheet so when we arrive at 9am we can open the spreadsheet and see what the figures were at 4am.

If this is possible can someone please explaing if i can get this as a trial and test it, if so please also explain how to set this up.

Thanks
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Post by Tech Support » Mon Oct 03, 2005 3:02 pm

This task is no problem for Internet Macros PRO or Scripting Edition. The full-featured 30-day trial version can be downloaded from http://www.iopus.com/download

In your case I especially recommend the following macros and scripts. They come very close to what you want to do:

Example Macros:

Demo-Extract-Table (Save table content to a text file in CSV format)
Demo-Loop-Csv-2-Web (Submit the content of a file to a website, start with LOOP instead of PLAY)

Online Demos: (http://www.iopus.com/iim/support)

1. Read/import values to a website from a file (0.77MB, Amazon.com)

Examples with batch files:

"Example-Batchfile.bat" (Batch file + macro, demonstrates the use of the command line interface. The same syntax as in the batch files can be used to run Macros from the Windows Task Scheduler.)

To schedule a certain macro to run at a certain hour, please use the Windows task scheduler ( http://www.iopus.com/guides/winscheduler.htm ) and the IIM command line interface (same commands as used in the batch files!)


Examples that make use of the Scripting Interface:

"extract-2-file.vbs" (uses wsh-extract-and-fill-part1 and wsh-extract-and-fill-part1 macros)
"extract-2-database.vbs" (store information from a website in a database)

Examples for other programming languages:

The VBS examples can be used with VB6 or VB.NET with almost no modification. The software also comes with a VB6 and a VB.NET example project. They are located in c:\program files\InternetMacros\Examples\VisualBasic

Note: We also develop custom macros and scripts for a fixed price. Prices. The price includes free support for this custom solution. If you are interested in this option, please provide us with more details about your project at http://www.iopus.com/service/contact/ and I can send you a quote with a fixed price in a few days.
Post Reply