Need data extraction to work in the "loop"

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

Need data extraction to work in the "loop"

by amaytut on Fri Mar 11, 2016 11:36 am

I am hoping someone could advise me on the script below, I am new to this. I am having 2 issues now: 1) It will do the data extraction part only if I do "play", without the loop. But I want it on a loop. I need it to pull the entry from a cell on my datasource, enter it on the website, search, and save the result if any, and then do it on a loop moving down each cell. 2) Also, I would like to either have the extraction from each loop saved into one file, or else not save it if there is no result. I am using the iMacros Browser (trial) in Windows, because I needed the Image wizard for the slider bar and text extraction. :D

VERSION BUILD=8881205 RECORDER=FX
TAB T=1
'CSV = Comma Separated Values in each line of the file
SET !DATASOURCE vietnam.csv
'Number of columns in the CSV file. This must be accurate!
SET !DATASOURCE_COLUMNS 1
'Start at line 2 to skip the header in the file
SET !LOOP 1
'Increase the current position in the file with each loop
SET !DATASOURCE_LINE {{!LOOP}}
URL GOTO=https://sanctionssearch.ofac.treas.gov/
TAG POS=1 TYPE=IMG ATTR=ID:Slider1_handleImage
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:ctl00$MainContent$Slider1 CONTENT=80
TAG POS=1 TYPE=INPUT:TEXT FORM=ID:aspnetForm ATTR=ID:ctl00_MainContent_txtLastName CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:aspnetForm ATTR=ID:ctl00_MainContent_btnSearch
TAG POS=1 TYPE=DIV ATTR=ID:ctl00_MainContent_pnlResults EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=Extract_{{!NOW:ddmmyy_hhnnss}}.csv
WAIT SECONDS=3
amaytut
 
Posts: 1
Joined: Wed Mar 09, 2016 8:21 am

Re: Need data extraction to work in the "loop"

by chivracq on Sat Mar 12, 2016 8:40 am

amaytut wrote:I am hoping someone could advise me on the script below, I am new to this. I am having 2 issues now:
1) It will do the data extraction part only if I do "play", without the loop. But I want it on a loop. I need it to pull the entry from a cell on my datasource, enter it on the website, search, and save the result if any, and then do it on a loop moving down each cell.
2) Also, I would like to either have the extraction from each loop saved into one file, or else not save it if there is no result.

I am using the iMacros Browser (trial) in Windows, because I needed the Image wizard for the slider bar and text extraction. :D

Code: Select all
VERSION BUILD=8881205 RECORDER=FX
TAB T=1
'CSV = Comma Separated Values in each line of the file
SET !DATASOURCE vietnam.csv
'Number of columns in the CSV file. This must be accurate!
SET !DATASOURCE_COLUMNS 1
'Start at line 2 to skip the header in the file
SET !LOOP 1
'Increase the current position in the file with each loop
SET !DATASOURCE_LINE {{!LOOP}}
URL GOTO=https://sanctionssearch.ofac.treas.gov/
TAG POS=1 TYPE=IMG ATTR=ID:Slider1_handleImage
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:ctl00$MainContent$Slider1 CONTENT=80
TAG POS=1 TYPE=INPUT:TEXT FORM=ID:aspnetForm ATTR=ID:ctl00_MainContent_txtLastName CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:aspnetForm ATTR=ID:ctl00_MainContent_btnSearch
TAG POS=1 TYPE=DIV ATTR=ID:ctl00_MainContent_pnlResults EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=Extract_{{!NOW:ddmmyy_hhnnss}}.csv
WAIT SECONDS=3

Okay..., I had a look at your Site and your Script, but FCIM...! :mrgreen: (Always mention your FCI when you open a Thread, I don't react to Threads otherwise...)

You mention using the iMB because of the Image Wizard, but I don't see any Image Recognition specific Statements in your Script, the Statement for the Slider works fine in FF and in all other Browsers very probably, and on FF, the EVENT Mode would work as well (+ iMB v11).

1- I think you get mixed up about the Extraction working only if you play your Macro once, the Extraction does work as well when looping your Script, but the '!EXTRACT_TEST_POPUP' gets automatically deactivated in Loop Mode.

2- If you want to save all Extracts in one single File, you just need to adapt the Format you are using for '{{!NOW}}', as you are currently including Min + Sec in your Filename, it's obvious that a a few Seconds later when doing the next Loop, the Time will have changed...

So I tested this Script and it works on FF using the iMacros for FF Add-on:
Code: Select all
VERSION BUILD=8961227 RECORDER=FX
'SET !EXTRACT_TEST_POPUP NO

TAB T=1

'CSV = Comma Separated Values in each line of the file
'SET !DATASOURCE vietnam.csv
'Number of columns in the CSV file. This must be accurate!
'SET !DATASOURCE_COLUMNS 1
'Start at line 2 to skip the header in the file
SET !LOOP 1
'Increase the current position in the file with each loop
'SET !DATASOURCE_LINE {{!LOOP}}

URL GOTO=https://sanctionssearch.ofac.treas.gov/
'TAG POS=1 TYPE=IMG ATTR=ID:Slider1_handleImage // (Not needed...)
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:ctl00$MainContent$Slider1 CONTENT=80
'WAIT SECONDS=1

'TAG POS=1 TYPE=INPUT:TEXT FORM=ID:aspnetForm ATTR=ID:ctl00_MainContent_txtLastName CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:TEXT FORM=ID:aspnetForm ATTR=ID:ctl00_MainContent_txtLastName CONTENT=Johnson
'PAUSE
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:aspnetForm ATTR=ID:ctl00_MainContent_btnSearch
TAG POS=1 TYPE=DIV ATTR=ID:ctl00_MainContent_pnlResults EXTRACT=TXT

'PAUSE
'SAVEAS TYPE=EXTRACT FOLDER=* FILE=Extract_{{!NOW:ddmmyy_hhnnss}}.csv
SAVEAS TYPE=EXTRACT FOLDER=* FILE=Extract_{{!NOW:ddmmyy_hh}}.csv
WAIT SECONDS=3
(Tested on iMacros for FF v8.8.2, Pale Moon v26.1.1 (=FF44), Win10-x64.)

Oh...!, and for saving your extracted Data only if you get some Result(s), you will need to use a "Conditional SAVEAS", like I've explained in this Thread:
- exclude extracted data from SAVEAS based on criteria ?
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6479
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)


Return to Data Extraction and Web Screen Scraping

Who is online

Users browsing this forum: Bing [Bot] and 2 guests

-->