Yahoo notepad extraction

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
rsw
Posts: 8
Joined: Tue Dec 06, 2011 10:56 am

Yahoo notepad extraction

Post by rsw » Tue Dec 06, 2011 11:06 am

I have just downloaded BUILD=7210419 and pasted in the published code for extracting Yahoo Notepad content to disk. The code is
VERSION BUILD=7031111 RECORDER=FX
TAB T=1
SET !EXTRACT_TEST_POPUP NO
SET !LOOP 1
URL GOTO=http://notepad.yahoo.com/
TAG POS={{!LOOP}} TYPE=A ATTR=HREF:*i=* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
TAG POS={{!LOOP}} TYPE=A ATTR=HREF:*i=*
TAG POS=1 TYPE=TEXTAREA ATTR=TXT:* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=D:\backup\notepad FILE=NOTEPAD-{{!VAR1}}.txt
BACK
WAIT SECONDS=2

It bombs with the following, after a few loops: RuntimeError: element A specified by HREF:*i=* was not found, line: 8 (Error code: -921)
I guess this is because Yahoo has revamped Notepad. Has anyone written an updated macro?
This is on Firefox 8.0.1 and XP/SP2
Thanks
Ron
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Wed Dec 07, 2011 12:59 pm

Hi Ron,

The macro doesn't work for me, probably because of my yahoo account settings. But here is a quick way to debug this: when you get the error, can you record a click on the link you want the macro to click and post here the resulting TAG command? Does it have a different type of id?

Best regards,
Daniel, iOpus Support
rsw
Posts: 8
Joined: Tue Dec 06, 2011 10:56 am

Re: Yahoo notepad extraction

Post by rsw » Sun Dec 11, 2011 1:42 am

Thanks, Daniel

I tried what I thought you meant by this, but I have probably misunderstood. Do you mean: click on the link whose content I am trying to grab after the macro has bombed? This does not produce anything new in the error message window. I need to go away and do lots of homework, I suspect, but I do appreciate any advice

Ron
Jordinou
Posts: 8
Joined: Sun Dec 11, 2011 12:53 pm

Re: Yahoo notepad extraction

Post by Jordinou » Sun Dec 11, 2011 1:11 pm

coucou s'il vous plait
je suis français et j'aurai besoin d'aide de personne doué
j'aimerai faire une macro qui mette 20 coms par article sur skyrock mais le problème j'y arrive pas à faute de leur mise jour.
Voilà ce que j'ai fait
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/blog/ADD-COMMENT ATTR=ID:body CONTENT=01
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:addComment ATTR=NAME:create&&VALUE:Poster

erreur indiquer → RuntimeError: element * specified by ID:comment was not found, line: 1 (Error code: -921)
merci énormément

hello please
I am French and I shall need help of nobody endowed
I shall like making a macro which puts 20 comments by article on skyrock but the problem I arrive there not in for lack of their stake in the daytime.


That is what
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/blog/ADD-COMMENT ATTR=ID:body CONTENT=01
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:addComment ATTR=NAME:create&&VALUE:Poster

I was mistaken to indicate → RuntimeError: element * specified by ID:comment was not found, line: 1 (Error code: -921)
Thank you enormously
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Mon Dec 12, 2011 10:21 pm

Hi Ron,
I tried what I thought you meant by this, but I have probably misunderstood. Do you mean: click on the link whose content I am trying to grab after the macro has bombed?
Yes, but first hit the 'Record' button, so that the recording starts :). But make sure your original macro is saved under a different name (not "#Current.iim") before you record this new one.

Best regards,
Daniel, iOpus Support
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Mon Dec 12, 2011 10:23 pm

Hi,
That is what
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/blog/ADD-COMMENT ATTR=ID:body CONTENT=01
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:addComment ATTR=NAME:create&&VALUE:Poster

I was mistaken to indicate → RuntimeError: element * specified by ID:comment was not found, line: 1 (Error code: -921)
Thank you enormously
Could you please post the whole macro, so that I can run it myself and reproduce the error? We will then try to help you with that!

Best regards,
Daniel, iOpus Support
rsw
Posts: 8
Joined: Tue Dec 06, 2011 10:56 am

Re: Yahoo notepad extraction

Post by rsw » Fri Dec 16, 2011 7:49 am

Thanks again, Daniel. Here's what appears in the new file, after clicking on 6 successive links to be scraped in Yahoo Notepad:

VERSION BUILD=7401110 RECORDER=FX
TAB T=1
URL GOTO=http://notepad.yahoo.com/
TAG POS=1 TYPE=SPAN ATTR=ID:FT_12
TAG POS=1 TYPE=SPAN ATTR=ID:NT_4397
TAG POS=1 TYPE=SPAN ATTR=ID:NT_4398
TAG POS=1 TYPE=SPAN ATTR=ID:NT_4399
TAG POS=1 TYPE=SPAN ATTR=ID:NT_4400
TAG POS=1 TYPE=SPAN ATTR=ID:NT_4401

But additional no error apart from the earlier one. I notice the build is different. Does this help?

Ron
Jordinou
Posts: 8
Joined: Sun Dec 11, 2011 12:53 pm

Re: Yahoo notepad extraction

Post by Jordinou » Sun Dec 18, 2011 9:58 pm

hello

Here is the macro complete

VERSION BUILD=7401110 RECORDER=FX
TAB T=1
URL GOTO=http://tendremelodie.skyrock.com/40.html
TAG POS=1 TYPE=A ATTR=TXT:Commenter
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/302461 ... DD_COMMENT ATTR=ID:body_3024617872 CONTENT=1
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024617872 ATTR=ID:comment_form_submit_3024617872
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/302461 ... DD_COMMENT ATTR=ID:body_3024617872 CONTENT=2
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024617872 ATTR=ID:comment_form_submit_3024617872
TAG POS=2 TYPE=A ATTR=TXT:Commenter
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/302461 ... DD_COMMENT ATTR=ID:body_3024618038 CONTENT=3
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024618038 ATTR=ID:comment_form_submit_3024618038
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/302461 ... DD_COMMENT ATTR=ID:body_3024618038 CONTENT=4
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024618038 ATTR=ID:comment_form_submit_3024618038
TAG POS=3 TYPE=A ATTR=TXT:Commenter
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/302461 ... DD_COMMENT ATTR=ID:body_3024618338 CONTENT=5
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024618338 ATTR=ID:comment_form_submit_3024618338
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/302461 ... DD_COMMENT ATTR=ID:body_3024618338 CONTENT=6
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024618338 ATTR=ID:comment_form_submit_3024618338

I would want that this macro works on everything skyrock.
Thanks to you, and good luck:)
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Thu Dec 22, 2011 1:06 am

Hello Jordinou,

I only changed 'commenter' to 'comment' and it worked perfectly. I guess the text is different depending on the language you're viewing the website in. This works for me ('commenter' replaced with 'comment'):

Code: Select all

VERSION BUILD=7401110 RECORDER=FX
TAB T=1
URL GOTO=http://tendremelodie.skyrock.com/40.html
TAG POS=1 TYPE=A ATTR=TXT:Comment
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/3024617872-Partir-ou-rester-Calogero.html?action=ADD_COMMENT ATTR=ID:body_3024617872 CONTENT=1
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024617872 ATTR=ID:comment_form_submit_3024617872
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/3024617872-Partir-ou-rester-Calogero.html?action=ADD_COMMENT ATTR=ID:body_3024617872 CONTENT=2
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024617872 ATTR=ID:comment_form_submit_3024617872
TAG POS=2 TYPE=A ATTR=TXT:Comment
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/3024618038-Le-bonheur-c-est-d-etre-heureux-ce-n-est-pas-de-faire-croire-aux.html?action=ADD_COMMENT ATTR=ID:body_3024618038 CONTENT=3
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024618038 ATTR=ID:comment_form_submit_3024618038
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/3024618038-Le-bonheur-c-est-d-etre-heureux-ce-n-est-pas-de-faire-croire-aux.html?action=ADD_COMMENT ATTR=ID:body_3024618038 CONTENT=4
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024618038 ATTR=ID:comment_form_submit_3024618038
TAG POS=3 TYPE=A ATTR=TXT:Comment
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/3024618338-For-You-Tracy-Chapman.html?action=ADD_COMMENT ATTR=ID:body_3024618338 CONTENT=5
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024618338 ATTR=ID:comment_form_submit_3024618338
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://tendremelodie.skyrock.com/3024618338-For-You-Tracy-Chapman.html?action=ADD_COMMENT ATTR=ID:body_3024618338 CONTENT=6
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:add_comment_form_3024618338 ATTR=ID:comment_form_submit_3024618338
Best regards,
Daniel, iOpus Support
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Thu Dec 22, 2011 1:19 am

rsw wrote:Thanks again, Daniel. Here's what appears in the new file, after clicking on 6 successive links to be scraped in Yahoo Notepad:

VERSION BUILD=7401110 RECORDER=FX
TAB T=1
URL GOTO=http://notepad.yahoo.com/
TAG POS=1 TYPE=SPAN ATTR=ID:FT_12
TAG POS=1 TYPE=SPAN ATTR=ID:NT_4397
TAG POS=1 TYPE=SPAN ATTR=ID:NT_4398
TAG POS=1 TYPE=SPAN ATTR=ID:NT_4399
TAG POS=1 TYPE=SPAN ATTR=ID:NT_4400
TAG POS=1 TYPE=SPAN ATTR=ID:NT_4401

But additional no error apart from the earlier one. I notice the build is different. Does this help?

Ron
Hi Ron,

Very sorry for the delay!! Oh, unfortunately it looks like you clicked the <span> around the link (see TYPE=SPAN)... Could you try again and see if precisely clicking the link will record the TYPE=A, just like in the initial macro?
I will try to respond faster this time!

Thanks,
Daniel, iOpus Support
rsw
Posts: 8
Joined: Tue Dec 06, 2011 10:56 am

Re: Yahoo notepad extraction

Post by rsw » Thu Dec 22, 2011 7:13 am

Hello again Daniel

The contents of "TYPE=?" depends on whether I click on the text or the icon next to it. In the latter case, I get
VERSION BUILD=7401110 RECORDER=FX
TAB T=1
URL GOTO=http://au.notepad.yahoo.com/
TAG POS=1 TYPE=DIV ATTR=ID:NI_4397
TAG POS=1 TYPE=DIV ATTR=ID:NI_4399
TAG POS=1 TYPE=DIV ATTR=ID:NI_4400
TAG POS=1 TYPE=DIV ATTR=ID:NI_4404
TAG POS=1 TYPE=IMG ATTR=SRC:http://l.yimg.com/a/i/brand/purplelogo/ ... d-n/au.gif

but I suspect this is not what you are looking for. Clicking on the text produces "TYPE=SPAN" as above. The text in question is the note name. Clicking on either icon or text produces the contents of the note in Notepad. The last line above is from clicking on the "Yahoo Notepad" page title.

If all this seems rather naive, you are right, and feel free to tell me to go away and learn more before asking more questions :)
Ron
Jordinou
Posts: 8
Joined: Sun Dec 11, 2011 12:53 pm

Re: Yahoo notepad extraction

Post by Jordinou » Thu Dec 22, 2011 6:42 pm

Hello :)
I would want that this maccro works on all the blogs before I had that there

TAG POS=1 TYPE=A ATTR=TXT:Comment
TAG POS=1 TYPE=TEXTAREA ATTR=ID:body CONTENT=01
TAG POS=1 TYPE=INPUT:SUBMIT ATTR=VALUE:Ajouter
TAG POS=1 TYPE=TEXTAREA ATTR=ID:body CONTENT=02
TAG POS=1 TYPE=INPUT:SUBMIT ATTR=VALUE:Ajouter
TAG POS=1 TYPE=TEXTAREA ATTR=ID:body CONTENT=03
TAG POS=1 TYPE=INPUT:SUBMIT ATTR=VALUE:Ajouter

But as you go to see she dashes not, I shall like that she puts comments on any article of a blog. Thank you in advance for your help...
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Thu Dec 22, 2011 11:04 pm

Hi Ron,

I'm very sorry, its so complex without seeing what you see :(. May be the solution would be to get a few screen shots from you on what the screen you're automating looks like and where the error occurs? I hope it'll help me to reproduce the issue, then I'll do my best to help!

Best regards,
Daniel, iOpus Support
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Thu Dec 22, 2011 11:06 pm

Hi Jordinou,

I didn't quite get the question - do you want to use the same macro to automate a different website/blog, not only the http://tendremelodie.skyrock.com/40.html?
Daniel, iOpus Support
rsw
Posts: 8
Joined: Tue Dec 06, 2011 10:56 am

Re: Yahoo notepad extraction

Post by rsw » Fri Dec 23, 2011 5:13 am

Hi Daniel
You are very patient. Here is a screenshot as I record a click on the icon and text for the note "EA login" (whose contents are deliberately rubbish):
http://www.amarina.com.au/screenshot/screenshot.JPG

Hope this helps
Ron
Post Reply