Yahoo notepad extraction

Discussions and Tech Support related to website data extraction, screen scraping and data mining using iMacros.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Jordinou
Posts: 8
Joined: Sun Dec 11, 2011 12:53 pm

Re: Yahoo notepad extraction

Post by Jordinou » Fri Dec 23, 2011 2:09 pm

Hi Daniel:)

Yes, which works on everything skyblogs example (tendremelodie, difool, lequipe-skyrock.....)
All the blogs of the skyrock site.
One universal maccro which we throw(launch) on any blog and work.
Thank you in advance:)
Last edited by Jordinou on Fri Dec 23, 2011 2:14 pm, edited 1 time in total.
Jordinou
Posts: 8
Joined: Sun Dec 11, 2011 12:53 pm

Re: Yahoo notepad extraction

Post by Jordinou » Fri Dec 23, 2011 2:10 pm

Hi Daniel:)

Yes, which works on everything skyblogs example (tendremelodie, difool, lequipe-skyrock)
All the blogs of the skyrock site.
One universal maccro which we throw(launch) on any blog and work.
Thank you in advance:)
Jordinou
Posts: 8
Joined: Sun Dec 11, 2011 12:53 pm

Re: Yahoo notepad extraction

Post by Jordinou » Fri Dec 23, 2011 2:11 pm

Hi Daniel:)

Yes, which works on everything skyblogs example (tendremelodie, difool, lequipe-skyrock)
All the blogs of the skyrock site.

One universal maccro which we throw(launch) on any blog and work.

Thank you in advance :)
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Fri Dec 23, 2011 4:17 pm

Hi Ron,

Thanks for the screen shot! Oh, you're asking valid questions, not those "do-it-for-me-immediately" ones :), so I'm really glad to help as much as I can.
I believe this is indeed because of the code change on Yahoo Notepad, but here is a question: do you really need to extract the lines one by one? If extracting the whole note is fine, here is what will do the job:

Code: Select all

TAG POS=1 TYPE=TEXTAREA ATTR=ID:txtContent EXTRACT=TXT
If however you do need the individual lines, let me know and I'll tell you how to do that using (JavaScript) scripting interface.

Best regards,
Daniel, iOpus Support
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Fri Dec 23, 2011 4:20 pm

Hi Jordinou,

I don't know how different those blogs are from each other. If their HTML is very different, then you might need a separate macro for each.
If you give me a link to one of those blogs, I will see whether these are compatible.

Best regards,
Daniel, iOpus Support
Jordinou
Posts: 8
Joined: Sun Dec 11, 2011 12:53 pm

Re: Yahoo notepad extraction

Post by Jordinou » Fri Dec 23, 2011 4:28 pm

Hi Daniel :)
Before, I had it make a who(which) to walk(work) that this, her(it) was universal to work on everything blog.....

TAG POS=1 TYPE=A ATTR=TXT:Commenter
TAB T=2
FRAME F=0
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://www.skyrock.com/blog/add_comment.php ATTR=ID:body CONTENT=O1
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ACTION:http://www.skyrock.com/blog/add_comment.php ATTR=VALUE:Ajouter
TAB T=2
TAB CLOSE
TAB T=1
TAG POS=1 TYPE=A ATTR=TXT:Commenter
TAB T=2
FRAME F=0
TAG POS=1 TYPE=TEXTAREA FORM=ACTION:http://www.skyrock.com/blog/add_comment.php ATTR=ID:body CONTENT=O2
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ACTION:http://www.skyrock.com/blog/add_comment.php ATTR=VALUE:Ajouter
TAB T=2
TAB CLOSE


But as you go to see him(it) she(it) does not work any more because skyrock have to change their version I shall want to have an identical of it but for the new version here is. thanks you
rsw
Posts: 8
Joined: Tue Dec 06, 2011 10:56 am

Re: Yahoo notepad extraction

Post by rsw » Fri Dec 23, 2011 10:31 pm

Thanks, Daniel - yes, that extracts the whole page of Notepad note titles, but not their content (ie what you see when you click on the title), so maybe that's why the original code used a loop?

BTW, now I have read a little, this is very neat software. I can see some other applications popping up even before I have this one going correctly :)
Ron
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Mon Jan 09, 2012 2:44 pm

Hi Ron,

Sorry, its my first time on the forum after holidays :(. Unfortunately it works fine on my end (extracts the content of the doc) and I'm not sure why it doesn't on yours...
Do you see any differences to your setting here that could be relevant:

Image

Best regards,
Daniel, iOpus Support
rsw
Posts: 8
Joined: Tue Dec 06, 2011 10:56 am

Re: Yahoo notepad extraction

Post by rsw » Mon Jan 09, 2012 9:32 pm

Hi Daniel
I am sure you deserved a holiday.
We are making progress. I switched to IE (previously Firefox) and can now extract individual Notes. I then tried to implement the original loop structure to extract all Notes, using:

VERSION BUILD=7031111 RECORDER=FX
TAB T=1
SET !EXTRACT_TEST_POPUP NO
SET !LOOP 1
URL GOTO=http://notepad.yahoo.com/
TAG POS=1 TYPE=TEXTAREA ATTR=ID:txtContent EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
TAG POS={{!LOOP}} TYPE=A ATTR=HREF:*i=*
TAG POS=1 TYPE=TEXTAREA ATTR=ID:txtContent EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=D:\backup\notepad FILE=NOTEPAD-{{!VAR1}}.txt
BACK
WAIT SECONDS=2

but got the following:
Error -1300: Cannot find HTML element of type "A:" with attribute(s) "HREF:*i=*".. Line 8: TAG POS={{!LOOP}}

I am struggling to understand the correct use of TAG
Many thanks
Ron
handlex
Posts: 1
Joined: Tue Jan 31, 2012 11:46 pm

Re: Yahoo notepad extraction

Post by handlex » Wed Feb 01, 2012 12:07 am

Well, I am by no means a fan of iOpus software after having to learn their incredibly annoying scripting language and associated gotchas... but I was still able to get the job done. I suppose I'm more annoyed at Yahoo for breaking notepad search and more or less ignoring my support ticket. Also the fact that Yahoo made their new notepad interface virtually unusable with no note sorting... and had no opt-out to return to the previous incarnation. Plus they did a terrible job coding the new one with an ungodly mix of a static page + javascript updates, with zero facility to link directly to a particular note.

Anyway, enough ranting, here's the code that I just used to move ~15 years of notes off Yahoo's server (you know, before they discontinue the service, which I think is their next move):

VERSION BUILD=7511734 RECORDER=FX
SET !LOOP 2
SET !REPLAYSPEED FAST
SET !TIMEOUT_PAGE 1
SET !TIMEOUT_STEP 1
SET !TIMEOUT_MACRO 60
SET !EXTRACT_TEST_POPUP NO
SET !EXTRACT NULL

'Simulate tag click and wait for update
TAG POS={{!LOOP}} TYPE=SPAN ATTR=ID:*NT_*
SET !EXTRACT NULL
WAIT SECONDS=3

'Get note name and drop illegal characters for filename
TAG POS={{!LOOP}} TYPE=SPAN ATTR=ID:*NT_* EXTRACT=TXT
SET !VAR1 EVAL("var s=\"{{!EXTRACT}}\"; s.replace(/[^a-zA-Z 0-9\\-]+/g,'')")
SET !EXTRACT NULL

'Get note contents
TAG POS=1 TYPE=TEXTAREA ATTR=ID:txtContent* EXTRACT=TXT
SAVEAS TYPE=EXTRACT FOLDER=c:\notepad FILE={{!VAR1}}.txt
SET !EXTRACT NULL
rsw
Posts: 8
Joined: Tue Dec 06, 2011 10:56 am

Re: Yahoo notepad extraction

Post by rsw » Wed Feb 01, 2012 1:45 am

Many thanks, handlex. It worked brilliantly. And after using Yahoo since 1997, I share your frustrations with them. I still don't see anything quite as convenient as Notepad for some purposes, though. Surely an opportunity for someone? Ronald
Daniel, Tech Support
Posts: 1483
Joined: Tue Jan 26, 2010 11:35 am

Re: Yahoo notepad extraction

Post by Daniel, Tech Support » Wed Feb 01, 2012 3:51 pm

Handlex, thanks a lot for your code!

Ronald, I'm really sorry I couldn't help much :( What I eventually came up with worked perfectly on my end, so I just couldn't reproduce your situation. I guess Yahoo returns different code based on IP/country.

Best wishes to both of you!
Daniel, iOpus Support
janib4all
Posts: 132
Joined: Wed Jul 21, 2010 6:44 am
Location: Karachi, Sindh, Pakistan
Contact:

Re: Yahoo notepad extraction

Post by janib4all » Sun Mar 25, 2012 8:46 am

I'm using iMacro to automate my personal and official stuff, its a great tool. I don't offer scripts/code but if anyone required my assistance in their works, i'm here for your assistance. SkP: Janib4all
Hire the BoT-fReeak!
botspecialist.blogspot.com
janib4all
Posts: 132
Joined: Wed Jul 21, 2010 6:44 am
Location: Karachi, Sindh, Pakistan
Contact:

Re: Yahoo notepad extraction

Post by janib4all » Tue Apr 10, 2012 4:35 am

iMacro is good for most cases but its not good when a extraction require logical decisions. I use it for normal usage as its easy & light weighted but for extreme cases, I use kapow robomaker, which is way beyond sophisticated than iMacro and offer every kind of stuff.

I'm twitting 1,000 (each twitter include 5 mentions) daily just because of kapow software. If I'd done it with twitter, it might wasn't fully automated as it is at the moment.
Hire the BoT-fReeak!
botspecialist.blogspot.com
Post Reply