Extract text from multiple url's

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

Extract text from multiple url's

by bry on Sun Aug 14, 2016 9:10 am

Good morning,

I am having a problem extracting text from multiple url's and saving to csv. I am using the code below, the url's are json returns and I am running in firefox.

Code: Select all
VERSION BUILD=8031994
TAB T=1
URL GOTO=https://maps.googleapis.com/maps/api/distancematrix/json?origins=Balmain+East,+NSW,+Australia&destinations=Sydney,+NSW,+Australia&key=??
TAG POS=1 TYPE=BODY ATTR=* EXTRACT=TXT
URL GOTO=https://maps.googleapis.com/maps/api/distancematrix/json?origins=Balmoral+Lake+Macquarie,+NSW,+Australia&destinations=Tweed+Heads,+NSW,+Australia&key=??
TAG POS=1 TYPE=BODY ATTR=* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=+_{{!NOW:yyyymmdd_hhnnss}}


Each work perfectly on their own but when put together return only a small portion of the text.

Help would be much appreciated. Thanks! :D
bry
 
Posts: 2
Joined: Sun Aug 14, 2016 8:59 am

Re: Extract text from multiple url's

by iimfun on Mon Aug 15, 2016 6:56 am

Hi,

When I play the macro you provided, it saves all the text and csv file is as follows
Code: Select all
"{
   ""destination_addresses"" : [],
   ""error_message"" : ""The provided API key is invalid."",
   ""origin_addresses"" : [],
   ""rows"" : [],
   ""status"" : ""REQUEST_DENIED""
}
","{
   ""destination_addresses"" : [],
   ""error_message"" : ""The provided API key is invalid."",
   ""origin_addresses"" : [],
   ""rows"" : [],
   ""status"" : ""REQUEST_DENIED""
}
"
iimfun
 
Posts: 238
Joined: Tue Jul 19, 2016 6:06 am

Re: Extract text from multiple url's

by bry on Mon Aug 15, 2016 10:29 pm

Hi,

Can I confirm you are running this through the firefox extension?

I am only running the free trial version of the product for testing but would not have thought that would make a difference.

The pop ups show as you describe as stated in my initial post.

When I run it as is, I get a return:

Code: Select all
"{",
"{"


Any thoughts??
bry
 
Posts: 2
Joined: Sun Aug 14, 2016 8:59 am

Re: Extract text from multiple url's

by iimfun on Mon Aug 15, 2016 11:37 pm

Yes, I'm running free extension 'iMacros for Firefox'.

As I can't reproduce your error, only the way with two 'SAVEAS' may be recommended to you so far
Code: Select all
TAB T=1
URL GOTO=https://maps.googleapis.com/maps/api/distancematrix/json?origins=Balmain+East,+NSW,+Australia&destinations=Sydney,+NSW,+Australia&key=??
TAG POS=1 TYPE=BODY ATTR=* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=+_{{!NOW:yyyymmdd}}
URL GOTO=https://maps.googleapis.com/maps/api/distancematrix/json?origins=Balmoral+Lake+Macquarie,+NSW,+Australia&destinations=Tweed+Heads,+NSW,+Australia&key=??
TAG POS=1 TYPE=BODY ATTR=* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=+_{{!NOW:yyyymmdd}}
iimfun
 
Posts: 238
Joined: Tue Jul 19, 2016 6:06 am


Return to Data Extraction and Web Screen Scraping

Who is online

Users browsing this forum: Google [Bot] and 9 guests

-->