I am trying to scrape data from the website: https://reports.myreca.ca/publicsearch.aspx
The table that is generated is an ASP.NET reportviewer table.
I am providing data source in a CSV file, and I plan on setting this to loop for the amount of records we have.
The problem is that the saved file format is less than diserable and near impossible to work with.
My script:
VERSION BUILD=12.0.501.6698
'Uses a Windows script to submit several datasets to a website, e. g. for filling an online database
TAB T=1
TAB CLOSEALLOTHERS
' Specify input file (if !COL variables are used, IIM automatically assume a CSV format of the input file
'CSV = Comma Separated Values in each line of the file
SET !DATASOURCE Address.csv
'Start at line 2 to skip the header in the file
SET !LOOP 2
'Increase the current position in the file with each loop
SET !DATASOURCE_LINE {{!LOOP}}
' Fill web form
URL GOTO=https://reports.myreca.ca/publicsearch.aspx
TAG POS=1 TYPE=INPUT:SUBMIT ATTR=NAME:Button1
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:TextBox2 CONTENT={{!COL1}}
WAIT SECONDS=3
TAG POS=1 TYPE=INPUT:SUBMIT ATTR=NAME:Button3
TAG POS=1 TYPE=TABLE ATTR=ID:ReportViewer1_fixedTable EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=* FILE=report.csv
--
In excel the entire output is saved on one cell and is really hard to work with. Is there a way that I can set it so the export is in somewhat workable format?
The output:
"
Legend for Sectors: Res. = Residential, Comm. = Commercial, PM. = Property Management, Rur. = Rural
Executed on: 8/14/2018 9:48:55 AM
Real Estate
Status
Licence History
First
Middle
Last
AKA
Brokerage
City
Class
Issue Date
Real Estate Sectors
Authorized
View
Greg
Alan
Steele
EXCELLENCE REAL ESTATE EDMONTON LTD. O/A RE/MAX EXCELLENCE
Edmonton
Associate
10/1/2017
Res. Comm. PM. Rur.
Authorized
View
Gregory
John
Steele
GREG
MOUNTAIN VIEW REAL ESTATE INC. O/A RE/MAX REAL ESTATE (MOUNTAIN VIEW)
Calgary
Associate
10/1/2017
Res. Comm. PM. Rur."
Extracting Data results in ugly output
Forum rules
iMacros EOL - Attention!
The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.
Thank you again for your business and support.
Sincerely,
The Progress Team
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
iMacros EOL - Attention!
The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.
Thank you again for your business and support.
Sincerely,
The Progress Team
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Re: Extracting Data results in ugly output
Hum..., some interesting Thread, finally, ah-ah....! (Sorry, it's Summertime, the Forum is a bit quiet at the moment, mostly Script-Kiddies trying to hack their Friends on the Beach...! )RAEALI wrote:I am trying to scrape data from the website: https://reports.myreca.ca/publicsearch.aspx
The table that is generated is an ASP.NET reportviewer table.
I am providing data source in a CSV file, and I plan on setting this to loop for the amount of records we have.
The problem is that the saved file format is less than diserable and near impossible to work with.
My script:--Code: Select all
VERSION BUILD=12.0.501.6698 'Uses a Windows script to submit several datasets to a website, e. g. for filling an online database TAB T=1 TAB CLOSEALLOTHERS ' Specify input file (if !COL variables are used, IIM automatically assume a CSV format of the input file 'CSV = Comma Separated Values in each line of the file SET !DATASOURCE Address.csv 'Start at line 2 to skip the header in the file SET !LOOP 2 'Increase the current position in the file with each loop SET !DATASOURCE_LINE {{!LOOP}} ' Fill web form URL GOTO=https://reports.myreca.ca/publicsearch.aspx TAG POS=1 TYPE=INPUT:SUBMIT ATTR=NAME:Button1 TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:TextBox2 CONTENT={{!COL1}} WAIT SECONDS=3 TAG POS=1 TYPE=INPUT:SUBMIT ATTR=NAME:Button3 TAG POS=1 TYPE=TABLE ATTR=ID:ReportViewer1_fixedTable EXTRACT=TXT SAVEAS TYPE=EXTRACT FOLDER=* FILE=report.csv
In excel the entire output is saved on one cell and is really hard to work with. Is there a way that I can set it so the export is in somewhat workable format?
The output:Code: Select all
" Legend for Sectors: Res. = Residential, Comm. = Commercial, PM. = Property Management, Rur. = Rural Executed on: 8/14/2018 9:48:55 AM Real Estate Status Licence History First Middle Last AKA Brokerage City Class Issue Date Real Estate Sectors Authorized View Greg Alan Steele EXCELLENCE REAL ESTATE EDMONTON LTD. O/A RE/MAX EXCELLENCE Edmonton Associate 10/1/2017 Res. Comm. PM. Rur. Authorized View Gregory John Steele GREG MOUNTAIN VIEW REAL ESTATE INC. O/A RE/MAX REAL ESTATE (MOUNTAIN VIEW) Calgary Associate 10/1/2017 Res. Comm. PM. Rur."
But, hum, could you first "clarify' a bit your FCI...? (Read my Sig...)
=> FCIM
=> v12.0: iMB v12.0 ('Full' or 'Trial') or iMacros for IE v12.0...?
+ OS...? (+ OS Language...? + Regional Settings...? (Used on iMB + IE for DataDelimiter/DataSeparator in '.CSV' Files...))
Hum, URL mentioned, perfect, but I cannot guess what you've got in your DataSource for '{{!COL1}}', if you can mention some valid Search Keyword, or post the direct URL to that Record...?
>>>
Hum, and mini-Badge earnt for being the 34th User on the Forum (ever!) to use the Word "Ugly", since 13 years the Forum exists...
(I'm still the Golden-Badge Owner with 25+ Posts (from 75 Posts in total), ah-ah...!)
Hum, 75x "ugly" out of 7500 Posts = 1%, hum..., still not my "favorite" Word then I guess, and I'm afraid "CIM" or "FCIM" beat "ugly" unfortunately by at least x50 times..., oops...!
>>>
And re-hum..., you could maybe edit your OP to use the ']CODE[' Forum Tags like I did in my Quote around your Script but especially around your ugly (oops, sorry => "long") '.CSV' Output that requires a lot of Scrolling, to ease a bit the Readability of the Thread...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
Re: Extracting Data results in ugly output
Hum, react a bit quickly if you expect some quick Answer/Solution, already 03h for me (NL-EUR), I'll soon be going to bed...
(And I would rather be very surprised if anybody else tried to answer your Thread..., or you'll have to wait until "my" tomorrow, ah-ah...! )
(And I would rather be very surprised if anybody else tried to answer your Thread..., or you'll have to wait until "my" tomorrow, ah-ah...! )
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
Re: Extracting Data results in ugly output
Hum, you don't seem to be in a hurry, looks like..., ah-ah...! Checked the Thread but no Follow-up (yet), hum, OK, fair enough...
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE'/'Trial').
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...
- FCI not mentioned: I don't even read the Qt...! (or only to catch Spam!)
- Script & URL help a lot for more "educated" Help...