Unable to download multiple images from website

Support for iMacros. The iMacros software is the unique solution for automating every activity inside a web browser, for data extraction and web testing.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
brg48
Posts: 14
Joined: Fri Sep 06, 2019 3:45 pm

Unable to download multiple images from website

Post by brg48 » Fri Apr 30, 2021 12:57 pm

I'm having trouble downloading images from a website where I need to run a few thousand records through and download the images that come up. I'm using:

iMacros Browser (x86) Version 12.5.503.8802
Released on 11/5/2018
Licensed Product: iMacros Professional Edition
Windows 10 home 64 bit

The website is: https://bcpa.net/RecID.asp

I'm entering various parcel numbers and trying to download each of the pictures that come up. Here are the various parcel numbers I'm testing: 494129-05-4220, 514026-01-0403, 504113-03-1410. I don't know how to change my script to adjust for the different references for each photo. I know very little about coding and I think this is a simple issue but it's beyond my ability.

Here is my script:

VERSION BUILD=12.5.503.8802
TAB T=1
TAB CLOSEALLOTHERS
'SET !PLAYBACKDELAY 0.43
SET !TIMEOUT_STEP 1
SET !ERRORIGNORE YES
'use a data source
SET !DATASOURCE Broward.csv
'start on the second record
SET !LOOP 2
'use the rest of the data in the data source?
SET !DATASOURCE_LINE {{!LOOP}}
URL GOTO=https://bcpa.net/RecID.asp
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:FOLIO_NUMBER CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:IMAGE ATTR=NAME:submit
TAG POS=1 TYPE=IMG ATTR=BORDER:0&&ALT:Photographs&&SRC:https://bcpa.net/Images/photographs.gif
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/ ... 163228.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_1
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/ ... 081550.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_2
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/ ... 083137.jpg" CONTENT=EVENT:SAVEITEM

Any help would be greatly appreciated.
chivracq
Posts: 9762
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Unable to download multiple images from website

Post by chivracq » Fri Apr 30, 2021 5:09 pm

brg48 wrote:
Fri Apr 30, 2021 12:57 pm
I'm having trouble downloading images from a website where I need to run a few thousand records through and download the images that come up. I'm using:

Code: Select all

iMacros Browser (x86) Version 12.5.503.8802
Released on 11/5/2018
Licensed Product: iMacros Professional Edition
Windows 10 home 64 bit
The website is: https://bcpa.net/RecID.asp

I'm entering various parcel numbers and trying to download each of the pictures that come up. Here are the various parcel numbers I'm testing: 494129-05-4220, 514026-01-0403, 504113-03-1410. I don't know how to change my script to adjust for the different references for each photo. I know very little about coding and I think this is a simple issue but it's beyond my ability.

Here is my script:

Code: Select all

VERSION BUILD=12.5.503.8802
TAB T=1
TAB CLOSEALLOTHERS
'SET !PLAYBACKDELAY 0.43
SET !TIMEOUT_STEP 1
SET !ERRORIGNORE YES
'use a data source
SET !DATASOURCE Broward.csv
'start on the second record
SET !LOOP 2
'use the rest of the data in the data source?
SET !DATASOURCE_LINE {{!LOOP}}
URL GOTO=https://bcpa.net/RecID.asp
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:FOLIO_NUMBER CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:IMAGE ATTR=NAME:submit
TAG POS=1 TYPE=IMG ATTR=BORDER:0&&ALT:Photographs&&SRC:https://bcpa.net/Images/photographs.gif
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PHOTOGRAPH-20170228163228.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_1
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20140320081550.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_2
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20111115083137.jpg" CONTENT=EVENT:SAVEITEM
Any help would be greatly appreciated.

Hum..., pay attention to open your Threads in the "correct" Sub-Forum, => would have been the 'Data-Extraction & Site Scraping' Sub-Forum for this one... :!:
That makes it also easier for other Users searching the Forum with a similar Qt/Scenario...
And hum, I "see" I should have mentioned it also for this previous Thread of yours, and this one on the contrary was opened in the 'Data-Extraction' Sub-Forum while it belongs to the 'General' one, ah-ah...!
But OK, don't do "anything", I'll ask the Forum Admin to "clean up the Mess" and to move those Threads to their "correct" Sub-Forum... :wink:

And Good Quality for this Thread, good...! :D
(Still haven't found the ']code[' Meta-Tags though, ah-ah...!)

>>>

Alright, yep-yep, typical Scenario with downloading Images, this is pretty straightforward to "solve", => you need to replace the "changing" Parts (usually in the URL's) by Wildcards and to use "POS=1" + "POS=2" + "POS=3" etc for the 'POS=n' to adapt to the Wildcards...

=> This Part of your Script:

Code: Select all

ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PHOTOGRAPH-20170228163228.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_1
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20140320081550.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_2
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20111115083137.jpg" CONTENT=EVENT:SAVEITEM
... will become:

Code: Select all

ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PHOTOGRAPH-20170228163228.jpg" // (Recorded)
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM
'>
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_1
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20140320081550.jpg" // (Recorded)
TAG POS=2 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM
'>
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_2
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20111115083137.jpg" // (Recorded)
TAG POS=3 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM
That's it...! Not tested but I expect it to work "as is"... 8)

>>>

And then a "little Trick", as you mention "a few thousand records", that would speed up the whole Process, is that this Part of your Script:

Code: Select all

URL GOTO=https://bcpa.net/RecID.asp
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:FOLIO_NUMBER CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:IMAGE ATTR=NAME:submit
TAG POS=1 TYPE=IMG ATTR=BORDER:0&&ALT:Photographs&&SRC:https://bcpa.net/Images/photographs.gif
... is actually loading 3 Pages in the Browser in order to "reach" the "Final Destination Page" which is the 'Photographs' Page corresponding to a Parcel Nb while this Page can be accessed directly with just 1 Page Loading by computing the URL directly, based on the Parcel Nb... :idea:

The URL of the specific 'Photographs' Page for Parcel Nb = "504113-03-1410" (that you used in your Script, stored in 'COL_1' from your DataSource) would be:

Code: Select all

https://bcpa.net/Photographs.asp?Folio=504113031410
... => The Parcel Nb is simply directly (re)used in the URL, and you only need to remove the 2 Dashes..., which would give, for example:

Code: Select all

'URL GOTO=https://bcpa.net/Photographs.asp?Folio=504113031410 // (Recorded)

'Compute 'Photographs' Page URL:
SET Folio_Nb EVAL("var c='{{!COL1}}'; var z=c.split('-').join(''); z;")
URL GOTO=https://bcpa.net/Photographs.asp?Folio={{Folio_Nb}}
I've tested it (manually) with the 3 Parcel Nb's you've provided, and that seems to work... 8)

=> Then your whole Script would become....:

Code: Select all

VERSION BUILD=12.5.503.8802
TAB T=1
TAB CLOSEALLOTHERS
'SET !PLAYBACKDELAY 0.43
SET !TIMEOUT_STEP 1
SET !ERRORIGNORE YES

'use a data source
SET !DATASOURCE Broward.csv
'start on the second record
SET !LOOP 2
'use the rest of the data in the data source?
SET !DATASOURCE_LINE {{!LOOP}}

'Search on 'Parcel-ID':
'URL GOTO=https://bcpa.net/RecID.asp
'TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:FOLIO_NUMBER CONTENT={{!COL1}}
'TAG POS=1 TYPE=INPUT:IMAGE ATTR=NAME:submit
'TAG POS=1 TYPE=IMG ATTR=BORDER:0&&ALT:Photographs&&SRC:https://bcpa.net/Images/photographs.gif

'Access a 'Photographs' Page directly instead of using the 'Search' (on 'Parcel-ID') Func from the Site:
'URL GOTO=https://bcpa.net/Photographs.asp?Folio=504113031410 // (Recorded)
'>
'Compute 'Photographs' Page URL:
SET Folio_Nb EVAL("var c='{{!COL1}}'; var z=c.split('-').join(''); z;")
URL GOTO=https://bcpa.net/Photographs.asp?Folio={{Folio_Nb}}

'Download the first 3 Photos:
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PHOTOGRAPH-20170228163228.jpg" // (Recorded)
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM
'>
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_1
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20140320081550.jpg" // (Recorded)
TAG POS=2 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM
'>
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_2
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20111115083137.jpg" // (Recorded)
TAG POS=3 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM

WAIT SECONDS=2
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
Post Reply