Extract text from multiple url's

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
bry
Posts: 2
Joined: Sun Aug 14, 2016 3:59 pm

Extract text from multiple url's

Post by bry » Sun Aug 14, 2016 4:10 pm

Good morning,

I am having a problem extracting text from multiple url's and saving to csv. I am using the code below, the url's are json returns and I am running in firefox.

Code: Select all

VERSION BUILD=8031994
TAB T=1
URL GOTO=https://maps.googleapis.com/maps/api/distancematrix/json?origins=Balmain+East,+NSW,+Australia&destinations=Sydney,+NSW,+Australia&key=??
TAG POS=1 TYPE=BODY ATTR=* EXTRACT=TXT
URL GOTO=https://maps.googleapis.com/maps/api/distancematrix/json?origins=Balmoral+Lake+Macquarie,+NSW,+Australia&destinations=Tweed+Heads,+NSW,+Australia&key=??
TAG POS=1 TYPE=BODY ATTR=* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=+_{{!NOW:yyyymmdd_hhnnss}}
Each work perfectly on their own but when put together return only a small portion of the text.

Help would be much appreciated. Thanks! :D
iimfun
Posts: 239
Joined: Tue Jul 19, 2016 1:06 pm

Re: Extract text from multiple url's

Post by iimfun » Mon Aug 15, 2016 1:56 pm

Hi,

When I play the macro you provided, it saves all the text and csv file is as follows

Code: Select all

"{
   ""destination_addresses"" : [],
   ""error_message"" : ""The provided API key is invalid."",
   ""origin_addresses"" : [],
   ""rows"" : [],
   ""status"" : ""REQUEST_DENIED""
}
","{
   ""destination_addresses"" : [],
   ""error_message"" : ""The provided API key is invalid."",
   ""origin_addresses"" : [],
   ""rows"" : [],
   ""status"" : ""REQUEST_DENIED""
}
"
bry
Posts: 2
Joined: Sun Aug 14, 2016 3:59 pm

Re: Extract text from multiple url's

Post by bry » Tue Aug 16, 2016 5:29 am

Hi,

Can I confirm you are running this through the firefox extension?

I am only running the free trial version of the product for testing but would not have thought that would make a difference.

The pop ups show as you describe as stated in my initial post.

When I run it as is, I get a return:

Code: Select all

"{",
"{"
Any thoughts??
iimfun
Posts: 239
Joined: Tue Jul 19, 2016 1:06 pm

Re: Extract text from multiple url's

Post by iimfun » Tue Aug 16, 2016 6:37 am

Yes, I'm running free extension 'iMacros for Firefox'.

As I can't reproduce your error, only the way with two 'SAVEAS' may be recommended to you so far

Code: Select all

TAB T=1
URL GOTO=https://maps.googleapis.com/maps/api/distancematrix/json?origins=Balmain+East,+NSW,+Australia&destinations=Sydney,+NSW,+Australia&key=??
TAG POS=1 TYPE=BODY ATTR=* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=+_{{!NOW:yyyymmdd}}
URL GOTO=https://maps.googleapis.com/maps/api/distancematrix/json?origins=Balmoral+Lake+Macquarie,+NSW,+Australia&destinations=Tweed+Heads,+NSW,+Australia&key=??
TAG POS=1 TYPE=BODY ATTR=* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=+_{{!NOW:yyyymmdd}}
Post Reply