How to extract this site

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

How to extract this site

by footyfacts on Sat Apr 16, 2016 4:56 am

I am finding it very difficult to work out how to re-iterate extracting for this page

1. goto http://www.vcglr.vic.gov.au/
2. click Find a current licence https://liquor.vcglr.vic.gov.au/alarm_i ... m_internet
3. click Search Now under Search Temporary Licences
4. input 90110000 into Temporary Licence Number
5. extract results

re-iterate for 90110001 ... 90113500

Recording & Playback does not seem to work I assume due to the hidden items
As a novice IMacro i don't have the skills to solve this
any help would be greatly appreciated

EDIT: I probably still don't have the skills to do this but I got it working enough

thanks
gregF
Last edited by footyfacts on Mon Apr 25, 2016 6:16 pm, edited 1 time in total.
footyfacts
 
Posts: 8
Joined: Tue Jul 02, 2013 4:48 am

Re: How to extract this site

by footyfacts on Mon Apr 25, 2016 6:13 pm

Relevant Info I missed (yes read FAQs)

VERSION BUILD=8970419 RECORDER=FX
FIREFOX=43.0.1
WIN XP SP3

Probably Less Than Elegant Solution but it works

Code: Select all
var ret
var licn = 90113830;

var CodeH = 'CODE:SET !EXTRACT_TEST_POPUP NO\n';

CodeA =         'TAG POS=1 TYPE=IMG ATTR=SRC:https://liquor.vcglr.vic.gov.au/alarm_internet/images/menu/search-temp.gif\n';

var CodeC =     'TAG POS=1 TYPE=INPUT:SUBMIT FORM=NAME:form1 ATTR=NAME:Submit\n';
CodeC = CodeC + 'TAG POS=2 TYPE=DIV ATTR=TXT:Venue* Extract=Txt\n';
CodeC = CodeC + 'SAVEAS TYPE=EXTRACT FOLDER=F:\\PC-Stuff\\Sendmail\\DanPages FILE=VCGLR.CSV\n';


for (var lic = 90113840; lic > licn; lic--)
{

var CodeB =         'TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:Licence_no CONTENT=' + lic + '\n';

var CodeE = CodeH + CodeA + CodeB + CodeC;

iimPlay(CodeE);

}

footyfacts
 
Posts: 8
Joined: Tue Jul 02, 2013 4:48 am

Re: How to extract this site

by chivracq on Tue Apr 26, 2016 4:37 am

footyfacts wrote:Relevant Info I missed (yes read FAQs)

Code: Select all
VERSION BUILD=8970419 RECORDER=FX
FIREFOX=43.0.1
WIN XP SP3


Probably Less Than Elegant Solution but it works

Code: Select all
var ret
var licn = 90113830;

var CodeH = 'CODE:SET !EXTRACT_TEST_POPUP NO\n';

CodeA =         'TAG POS=1 TYPE=IMG ATTR=SRC:https://liquor.vcglr.vic.gov.au/alarm_internet/images/menu/search-temp.gif\n';

var CodeC =     'TAG POS=1 TYPE=INPUT:SUBMIT FORM=NAME:form1 ATTR=NAME:Submit\n';
CodeC = CodeC + 'TAG POS=2 TYPE=DIV ATTR=TXT:Venue* Extract=Txt\n';
CodeC = CodeC + 'SAVEAS TYPE=EXTRACT FOLDER=F:\\PC-Stuff\\Sendmail\\DanPages FILE=VCGLR.CSV\n';


for (var lic = 90113840; lic > licn; lic--)
{

var CodeB =         'TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:Licence_no CONTENT=' + lic + '\n';

var CodeE = CodeH + CodeA + CodeB + CodeC;

iimPlay(CodeE);

}


Hum..., we finally have your FCI..., good...! So you are on FF, then yep, you can use some '.js' Script with on the fly generated Macro(s) like the Script you've posted...

Here is the Script I had quickly made and posted but removed 1 day later, after you had reported my previous Post...:
Code: Select all
VERSION BUILD=8820413 RECORDER=FX
TAB T=1
SET !TIMEOUT_STEP 1
SET !EXTRACT_TEST_POPUP NO

URL GOTO=https://liquor.vcglr.vic.gov.au/alarm_internet/alarm_internet.ASP?WCI=index_action&WCU
TAG POS=1 TYPE=P ATTR=TXT:Search<SP>Temporary<SP>Licences
'TAG POS=1 TYPE=IMG ATTR=SRC:https://liquor.vcglr.vic.gov.au/alarm_internet/images/search-now-over.gif
TAG POS=R1 TYPE=IMG ATTR=SRC:https://liquor.vcglr.vic.gov.au/alarm_internet/images/search-now*.gif

TAG POS=1 TYPE=H1 ATTR=TXT:Search<SP>Temporary<SP>Licences
TAG POS=1 TYPE=P ATTR=TXT:Temporary<SP>Licence<SP>Number

SET Licence_Nr 90110000
ADD Licence_Nr -1
ADD Licence_Nr {{!LOOP}}
'TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:form1 ATTR=NAME:Licence_no CONTENT=90110000
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:form1 ATTR=NAME:Licence_no CONTENT={{Licence_Nr}}
TAG POS=1 TYPE=INPUT:SUBMIT FORM=NAME:form1 ATTR=NAME:Submit
TAG POS=1 TYPE=H1 ATTR=TXT:Search<SP>Results
TAG POS=1 TYPE=H2 ATTR=TXT:Temporary<SP>Licences
'>
'TAG POS=1 TYPE=DIV ATTR=TXT:Inactive<SP>Temporary<SP>Licences
TAG POS=R1 TYPE=DIV ATTR=TXT:* EXTRACT=TXT
'>
'TAG POS=2 TYPE=DIV ATTR=TXT:Venue<SP>Name:<SP>BALLARAT<SP>KART<SP>CLUB<SP>Venue<SP>Addre*
TAG POS=R1 TYPE=DIV ATTR=TXT:Venue<SP>Name:* EXTRACT=TXT

PROMPT Loop:<SP>{{!LOOP}}<SP>/<SP>Licence_Nr:<SP>{{Licence_Nr}}<BR><BR>{{!EXTRACT}}
(Tested on iMacros for FF v8.8.2, Pale Moon v26.1.1 (=FF44), Win10-x64.)
Script is generic and should work in all Browsers...

+ Use 'EVAL()' (+ 'split()' and 'trim())') if you want to get rid of the "garbage" Data about the 'View Licence' Button...

Result of the PROMPT/Extract:
Code: Select all
Loop: 1 / Licence_Nr: 90110000

Inactive Temporary Licences[EXTRACT]
Venue Name:
BALLARAT KART CLUB
Venue Address: 70 RACECOURSE RD HADDON 3351
Applicant: BALLARAT KARTING CLUB INC.
Application Number: 15L08790
Received Date: 12/11/2015 
Grant Date: 16/11/2015
Limited Licence: 90110000 Already expired on 29/11/2015

   
   




      <!--
      var sr = new submitroll("images/view-licence.gif","images/view-licence-over.gif","viewlicence_1");
      sr.write();
      //-->
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
chivracq
 
Posts: 6473
Joined: Sat Apr 13, 2013 6:07 am
Location: Amsterdam (NL)


Return to Data Extraction and Web Screen Scraping

Who is online

Users browsing this forum: No registered users and 2 guests

-->