Extracting data from www sites

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

Extracting data from www sites

by Pre sales tech question on Mon Oct 03, 2005 7:12 am

Hi, was woundering if some can assist us.

We have a requirement and would like to know if this application can help us.

Heres what we would like :

Every morning at 4am we would like some sort of program to visit a number of web site and get the figures for us, we would like them exported into a Excel sheet so when we arrive at 9am we can open the spreadsheet and see what the figures were at 4am.

If this is possible can someone please explaing if i can get this as a trial and test it, if so please also explain how to set this up.

Thanks
Pre sales tech question
 

by Tech Support on Mon Oct 03, 2005 8:02 am

This task is no problem for Internet Macros PRO or Scripting Edition. The full-featured 30-day trial version can be downloaded from http://www.iopus.com/download

In your case I especially recommend the following macros and scripts. They come very close to what you want to do:

Example Macros:

Demo-Extract-Table (Save table content to a text file in CSV format)
Demo-Loop-Csv-2-Web (Submit the content of a file to a website, start with LOOP instead of PLAY)

Online Demos: (http://www.iopus.com/iim/support)

1. Read/import values to a website from a file (0.77MB, Amazon.com)

Examples with batch files:

"Example-Batchfile.bat" (Batch file + macro, demonstrates the use of the command line interface. The same syntax as in the batch files can be used to run Macros from the Windows Task Scheduler.)

To schedule a certain macro to run at a certain hour, please use the Windows task scheduler ( http://www.iopus.com/guides/winscheduler.htm ) and the IIM command line interface (same commands as used in the batch files!)


Examples that make use of the Scripting Interface:

"extract-2-file.vbs" (uses wsh-extract-and-fill-part1 and wsh-extract-and-fill-part1 macros)
"extract-2-database.vbs" (store information from a website in a database)

Examples for other programming languages:

The VBS examples can be used with VB6 or VB.NET with almost no modification. The software also comes with a VB6 and a VB.NET example project. They are located in c:\program files\InternetMacros\Examples\VisualBasic

Note: We also develop custom macros and scripts for a fixed price. Prices. The price includes free support for this custom solution. If you are interested in this option, please provide us with more details about your project at http://www.iopus.com/service/contact/ and I can send you a quote with a fixed price in a few days.
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm


Return to Data Extraction and Web Screen Scraping

Who is online

Users browsing this forum: Google [Bot] and 3 guests

-->