Unable to download multiple images from website

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
iMacros EOL - Attention!

The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.

Thank you again for your business and support.

Sincerely,
The Progress Team

Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
brg48
Posts: 17
Joined: Fri Sep 06, 2019 3:45 pm

Unable to download multiple images from website

Post by brg48 » Fri Apr 30, 2021 12:57 pm

I'm having trouble downloading images from a website where I need to run a few thousand records through and download the images that come up. I'm using:

iMacros Browser (x86) Version 12.5.503.8802
Released on 11/5/2018
Licensed Product: iMacros Professional Edition
Windows 10 home 64 bit

The website is: https://bcpa.net/RecID.asp

I'm entering various parcel numbers and trying to download each of the pictures that come up. Here are the various parcel numbers I'm testing: 494129-05-4220, 514026-01-0403, 504113-03-1410. I don't know how to change my script to adjust for the different references for each photo. I know very little about coding and I think this is a simple issue but it's beyond my ability.

Here is my script:

VERSION BUILD=12.5.503.8802
TAB T=1
TAB CLOSEALLOTHERS
'SET !PLAYBACKDELAY 0.43
SET !TIMEOUT_STEP 1
SET !ERRORIGNORE YES
'use a data source
SET !DATASOURCE Broward.csv
'start on the second record
SET !LOOP 2
'use the rest of the data in the data source?
SET !DATASOURCE_LINE {{!LOOP}}
URL GOTO=https://bcpa.net/RecID.asp
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:FOLIO_NUMBER CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:IMAGE ATTR=NAME:submit
TAG POS=1 TYPE=IMG ATTR=BORDER:0&&ALT:Photographs&&SRC:https://bcpa.net/Images/photographs.gif
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/ ... 163228.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_1
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/ ... 081550.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_2
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/ ... 083137.jpg" CONTENT=EVENT:SAVEITEM

Any help would be greatly appreciated.
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Unable to download multiple images from website

Post by chivracq » Fri Apr 30, 2021 5:09 pm

brg48 wrote:
Fri Apr 30, 2021 12:57 pm
I'm having trouble downloading images from a website where I need to run a few thousand records through and download the images that come up. I'm using:

Code: Select all

iMacros Browser (x86) Version 12.5.503.8802
Released on 11/5/2018
Licensed Product: iMacros Professional Edition
Windows 10 home 64 bit
The website is: https://bcpa.net/RecID.asp

I'm entering various parcel numbers and trying to download each of the pictures that come up. Here are the various parcel numbers I'm testing: 494129-05-4220, 514026-01-0403, 504113-03-1410. I don't know how to change my script to adjust for the different references for each photo. I know very little about coding and I think this is a simple issue but it's beyond my ability.

Here is my script:

Code: Select all

VERSION BUILD=12.5.503.8802
TAB T=1
TAB CLOSEALLOTHERS
'SET !PLAYBACKDELAY 0.43
SET !TIMEOUT_STEP 1
SET !ERRORIGNORE YES
'use a data source
SET !DATASOURCE Broward.csv
'start on the second record
SET !LOOP 2
'use the rest of the data in the data source?
SET !DATASOURCE_LINE {{!LOOP}}
URL GOTO=https://bcpa.net/RecID.asp
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:FOLIO_NUMBER CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:IMAGE ATTR=NAME:submit
TAG POS=1 TYPE=IMG ATTR=BORDER:0&&ALT:Photographs&&SRC:https://bcpa.net/Images/photographs.gif
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PHOTOGRAPH-20170228163228.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_1
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20140320081550.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_2
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20111115083137.jpg" CONTENT=EVENT:SAVEITEM
Any help would be greatly appreciated.

Hum..., pay attention to open your Threads in the "correct" Sub-Forum, => would have been the 'Data-Extraction & Site Scraping' Sub-Forum for this one... :!:
That makes it also easier for other Users searching the Forum with a similar Qt/Scenario...
And hum, I "see" I should have mentioned it also for this previous Thread of yours, and this one on the contrary was opened in the 'Data-Extraction' Sub-Forum while it belongs to the 'General' one, ah-ah...!
But OK, don't do "anything", I'll ask the Forum Admin to "clean up the Mess" and to move those Threads to their "correct" Sub-Forum... :wink:

And Good Quality for this Thread, good...! :D
(Still haven't found the ']code[' Meta-Tags though, ah-ah...!)

>>>

Alright, yep-yep, typical Scenario with downloading Images, this is pretty straightforward to "solve", => you need to replace the "changing" Parts (usually in the URL's) by Wildcards and to use "POS=1" + "POS=2" + "POS=3" etc for the 'POS=n' to adapt to the Wildcards...

=> This Part of your Script:

Code: Select all

ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PHOTOGRAPH-20170228163228.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_1
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20140320081550.jpg" CONTENT=EVENT:SAVEITEM
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_2
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20111115083137.jpg" CONTENT=EVENT:SAVEITEM
... will become:

Code: Select all

ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PHOTOGRAPH-20170228163228.jpg" // (Recorded)
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM
'>
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_1
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20140320081550.jpg" // (Recorded)
TAG POS=2 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM
'>
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_2
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20111115083137.jpg" // (Recorded)
TAG POS=3 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM
That's it...! Not tested but I expect it to work "as is"... 8)

>>>

And then a "little Trick", as you mention "a few thousand records", that would speed up the whole Process, is that this Part of your Script:

Code: Select all

URL GOTO=https://bcpa.net/RecID.asp
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:FOLIO_NUMBER CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:IMAGE ATTR=NAME:submit
TAG POS=1 TYPE=IMG ATTR=BORDER:0&&ALT:Photographs&&SRC:https://bcpa.net/Images/photographs.gif
... is actually loading 3 Pages in the Browser in order to "reach" the "Final Destination Page" which is the 'Photographs' Page corresponding to a Parcel Nb while this Page can be accessed directly with just 1 Page Loading by computing the URL directly, based on the Parcel Nb... :idea:

The URL of the specific 'Photographs' Page for Parcel Nb = "504113-03-1410" (that you used in your Script, stored in 'COL_1' from your DataSource) would be:

Code: Select all

https://bcpa.net/Photographs.asp?Folio=504113031410
... => The Parcel Nb is simply directly (re)used in the URL, and you only need to remove the 2 Dashes..., which would give, for example:

Code: Select all

'URL GOTO=https://bcpa.net/Photographs.asp?Folio=504113031410 // (Recorded)

'Compute 'Photographs' Page URL:
SET Folio_Nb EVAL("var c='{{!COL1}}'; var z=c.split('-').join(''); z;")
URL GOTO=https://bcpa.net/Photographs.asp?Folio={{Folio_Nb}}
I've tested it (manually) with the 3 Parcel Nb's you've provided, and that seems to work... 8)

=> Then your whole Script would become....:

Code: Select all

VERSION BUILD=12.5.503.8802
TAB T=1
TAB CLOSEALLOTHERS
'SET !PLAYBACKDELAY 0.43
SET !TIMEOUT_STEP 1
SET !ERRORIGNORE YES

'use a data source
SET !DATASOURCE Broward.csv
'start on the second record
SET !LOOP 2
'use the rest of the data in the data source?
SET !DATASOURCE_LINE {{!LOOP}}

'Search on 'Parcel-ID':
'URL GOTO=https://bcpa.net/RecID.asp
'TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:FOLIO_NUMBER CONTENT={{!COL1}}
'TAG POS=1 TYPE=INPUT:IMAGE ATTR=NAME:submit
'TAG POS=1 TYPE=IMG ATTR=BORDER:0&&ALT:Photographs&&SRC:https://bcpa.net/Images/photographs.gif

'Access a 'Photographs' Page directly instead of using the 'Search' (on 'Parcel-ID') Func from the Site:
'URL GOTO=https://bcpa.net/Photographs.asp?Folio=504113031410 // (Recorded)
'>
'Compute 'Photographs' Page URL:
SET Folio_Nb EVAL("var c='{{!COL1}}'; var z=c.split('-').join(''); z;")
URL GOTO=https://bcpa.net/Photographs.asp?Folio={{Folio_Nb}}

'Download the first 3 Photos:
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PHOTOGRAPH-20170228163228.jpg" // (Recorded)
TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM
'>
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_1
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20140320081550.jpg" // (Recorded)
TAG POS=2 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM
'>
ONDOWNLOAD FOLDER=C:\Users\trey_\OneDrive\Documents\iMacros\Downloads\Florida\Broward FILE={{!COL1}}_2
'TAG POS=1 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/504113/03/1410/PUBLIC%20PHOTO-20111115083137.jpg" // (Recorded)
TAG POS=3 TYPE=IMG ATTR=HREF:"https://bcpa.net/Photographs/*.jpg" CONTENT=EVENT:SAVEITEM

WAIT SECONDS=2
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
chivracq
Posts: 10301
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Unable to download multiple images from website

Post by chivracq » Fri Sep 24, 2021 11:23 am

Kirito01 wrote:
Fri Sep 24, 2021 10:26 am
Kirito01 wrote:
Fri Sep 24, 2021 9:31 am
EDITED BY MOD (@chivracq): :!:
Grrr, some stupid Spammer who managed to "trick me" into approving their Post..., using some "interesting" Technique... :shock: :roll:
... Waiting for Forum Admin to delete the @User's Account... :!:
hahaha, man, you got me...but the thing i said about you, were true...i like your style..how you respond to other..amazing..including this one. no link this time. :P

Yeah, well, Thanks in a way for the "original" Compliment, but "too bad" you tried to spam the Forum, you "screwed it" a bit for yourself, I've already asked the Forum Admin to delete your Account, I don't want to have to keep "monitoring" you and checking and rechecking regularly your Posts for "invisible" Spam..., tja...! :|
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
Post Reply