Help with download of stock data

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information:CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
H@rry
Posts: 2
Joined: Sat Jun 22, 2019 7:17 pm

Help with download of stock data

Post by H@rry » Sat Jun 22, 2019 7:58 pm

I want to download data from two pages into two different files. These are tables containing stock market data. After I get the downloaded data I will further process it with Excel Macros and VB6 coding.
IMacro 9.0.3
Firefox 55.0.3
Windows 10

<code>: (Sorry but I don't see the item to include code)

VERSION BUILD=10.4.28.1074
TAB T=1
TAB CLOSEALLOTHERS
SET !EXTRACT_TEST_POPUP NO
TAG POS=1 TYPE=TABLE ATTR=TXT:* EXTRACT=TXT
URL GOTO=https://finance.yahoo.com/portfolio/p_57/view/view_8
WAIT SECONDS=6
SAVEAS TYPE=EXTRACT FOLDER=c:/stox17/{{!NOW:mm-dd-yyyy}}/ FILE=download_AJ.csv
TAG POS=1 TYPE=TABLE ATTR=TXT:* EXTRACT=TXT
URL GOTO=https://finance.yahoo.com/portfolio/p_93/view/view_8
WAIT SECONDS=6
SAVEAS TYPE=EXTRACT FOLDER=c:/stox17/{{!NOW:mm-dd-yyyy}}/ FILE=download_KZ.csv

</code>

I have two issues with the downloaded data.
1. The data is going into the wrong file. The data from the first page is going into the file with suffix KZ and the data from the second page is going into the file with the suffix AJ. I could simply rename the files but I don't think I should have to.
2. The first column has the data duplicated in the same column: ie "ABC" comes through as "ABCABC". Again I could handle that in programming but I don't think I should have to.

Thanx for any help and guidance.
chivracq
Posts: 8525
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Help with download of stock data

Post by chivracq » Sat Jun 22, 2019 8:32 pm

H@rry wrote:
Sat Jun 22, 2019 7:58 pm
I want to download data from two pages into two different files. These are tables containing stock market data. After I get the downloaded data I will further process it with Excel Macros and VB6 coding.

Code: Select all

IMacro 9.0.3
Firefox 55.0.3
Windows 10
<code>: (Sorry but I don't see the item to include code)

Code: Select all

VERSION BUILD=10.4.28.1074
TAB T=1
TAB CLOSEALLOTHERS
SET !EXTRACT_TEST_POPUP NO
TAG POS=1 TYPE=TABLE ATTR=TXT:* EXTRACT=TXT
URL GOTO=https://finance.yahoo.com/portfolio/p_57/view/view_8
WAIT SECONDS=6
SAVEAS TYPE=EXTRACT FOLDER=c:/stox17/{{!NOW:mm-dd-yyyy}}/ FILE=download_AJ.csv
TAG POS=1 TYPE=TABLE ATTR=TXT:* EXTRACT=TXT
URL GOTO=https://finance.yahoo.com/portfolio/p_93/view/view_8
WAIT SECONDS=6
SAVEAS TYPE=EXTRACT FOLDER=c:/stox17/{{!NOW:mm-dd-yyyy}}/ FILE=download_KZ.csv
</code>

I have two issues with the downloaded data.
1. The data is going into the wrong file. The data from the first page is going into the file with suffix KZ and the data from the second page is going into the file with the suffix AJ. I could simply rename the files but I don't think I should have to.
2. The first column has the data duplicated in the same column: ie "ABC" comes through as "ABCABC". Again I could handle that in programming but I don't think I should have to.

Thanx for any help and guidance.
You need to use Square Brackets for the Forum Metatags for ']CODE[' Blocks... (Or 5th Icon from the Editor, next to the ']QUOTE[' Icon...)

1- Yep, normal, you need to fire the 'EXTRACT' after the Page has loaded, and not before like you are currently doing, or the 'EXTRACT' will attempt to extract what "it can find" on the Page where the Script is already on.
Temporarily re-enable '!EXTRACT_TEST_POPUP' and you'll understand what I mean...

2- I don't now, could not access those 'p_57' + 'p_93' Pages to have a look, I guess those Pages are only available after being logged in...
(And what an "annoying" Site about their Cookie Management and "fake" Control & Opt-out... :shock: )
But if the 1st Col is duplicated in the 'EXTRACT' + ''SAVEAS', that's probably the way the 'TABLE' HTML Element on those Pages is constructed/populated by the Page/Site itself...
It will probably be possible to "clean" the Data from that "double" Col using 'EVAL()', or you'd need to extract the whole Table at the 'Cell' Level one by one if the 'EXTRACT' on the 'TYPE=TABLE' is "too powerful"...

>>>

Correct Spelling is "iMacros" btw... :wink:

And rather than using v9.0.3 for FF, "advised" Version is v8.9.7 for FF (available for Download from the iMacros for FF Legacy Wiki-Page) which is much more stable than (and not buggy and limited like) v9.0.3... (Both Versions work on FF v55.0.3 which is already the "advised" FF Version.)

Hum, and the Double Col could actually be related to v9.0.3 for FF, ah-ah...! I think I remember a few Threads about Double Data and Double Extracts when v9.0.3 got released... (I never used/installed that Version myself, too buggy...)
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
H@rry
Posts: 2
Joined: Sat Jun 22, 2019 7:17 pm

Re: Help with download of stock data

Post by H@rry » Sun Jun 23, 2019 4:18 am

I made the changes you suggested and installed v 8.9.7. That did the job, thanx!
chivracq
Posts: 8525
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: Help with download of stock data

Post by chivracq » Sun Jun 23, 2019 4:23 am

H@rry wrote:
Sun Jun 23, 2019 4:18 am
I made the changes you suggested and installed v 8.9.7. That did the job, thanx!
Ah OK, good to hear..., and Thanks for the Feedback/Follow-up... :D

Then, OK, the Double Col/Extract was indeed related to v9.0.3 then, ah-ah...!, LOL...! :P
- (F)CIM = (Full) Config Info Missing: iMacros + Browser + OS with all 3 Versions...
- I usually don't even read the Question if that (required) Info is not mentioned...
- Script & URL usually help a lot for a more "educated" Help...
Post Reply