How to automate Firefox with iimInit ("-fx")

Discussions and Tech Support related to automating the iMacros Browser or Internet Explorer from any scripting and programming language, such as VBS (WSH), VBA, VB, Perl, Delphi, C# or C++.

Moderators: Community Moderators, iMacros Moderators

Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the Google search box (at the top of each forum page) to see if a similar problem or question has already been addressed. This will search the entire contents of the forums as well as the iMacros Wiki.
3. We can respond much faster to your posts if you include the following information:

CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST

Answering your own posts (e.g. attempting to "bump" your topic) drops your topic from the list of unanswered threads, so it may actually receive less views.

Re: How to automate Firefox with iimInit ("-fx")

by Tom, Tech Support on Tue Feb 22, 2011 12:13 pm

Hello echan00,

Would you please describe exactly what problem you are experiencing. This thread (and the other thread you previously posted too) pertain primarily to using the iMacros Scripting Interface to automate Firefox.

There shouldn't be any problems with using the built-in Javascript scripting interface in iMacros for Firefox.
Regards,

Tom, iMacros Support
Tom, Tech Support
 
Posts: 3298
Joined: Mon May 31, 2010 9:59 am

Re: How to automate Firefox with iimInit ("-fx")

by Tech Support on Wed Mar 02, 2011 9:22 am

Update: iMacros V7.22 and iMacros for Firefox 7.1.1.5 solve the issue reported by mjayson.
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm

Re: How to automate Firefox with iimInit ("-fx")

by winxp4all on Tue Mar 08, 2011 2:26 am

Windows version = Windows 7 Ultimate, 32-bit, Service Pack 1
iMacros scripting version = 7.22.1254
Firefox version = 3.6.15
Firefox iMacros plug-in version = 7.1.1.5
Python version 3.2 with win32com.client build 216

There are many serious problems:

1. iimInit("-fx",False) does not work, always starts a new Firefox session.
2. iimInit("-fx") followed by iimPlay("CODE: URL GOTO=...") does not work. The iimPlay() is sent before the browser comes up and therefore gets lost.
3. iimGetLastExtract(), iimGetLastExtract(0), iimGetLastExtract(1) always returns "OK (1)" if there is only one extract statement. If there are 2 extract statements, iimGetLastExtract(), iimGetLastExtract(0) returns the data but iimGetLastExtract(1) and iimGetLastExtract(2) returns "OK (1)" again.
4. With the iMacros browser (and IE also)
iimPlay("CODE: TAG POS=1 ....") followed by iimPlay("CODE: TAG POS=R1 ...") does not work. The anchor somehow was lost so the following relative TAG failed. The same sequence works fine if they are sent together.
iimPlay("CODE: TAG POS=1 ....") followed by iimPlay("CODE: TAG POS=R1 ...") works fine with Firefox so relative tagging with iimPlay() must be OK. It just does not work with iMacros browser and IE.
5. Performance seems to be quite slow: the Firefox javascript version is significantly faster than the remote control scripting version, executing the same sequence of iimPlay("CODE: TAG ...") statements. Lots of CPU cycles are consumed by the remote control version as compared to the javascript version.

It looks like the current iMacros scripting version is far from stable for production work.
winxp4all
 
Posts: 9
Joined: Tue Mar 08, 2011 1:09 am

Re: How to automate Firefox with iimInit ("-fx")

by Tech Support on Tue Mar 08, 2011 5:03 am

Thanks you for the system details. They look all ok to me.

1. iimInit("-fx",False) does not work, always starts a new Firefox session.

Yes, FALSE is not supported for Firefox. This parameter usually is not required for unattended automation. If you have a special use case that would require please let us know.

2. iimInit("-fx") followed by iimPlay("CODE: URL GOTO=...") does not work. The iimPlay() is sent before the browser comes up and therefore gets lost.

That works fine in our test. As a test, does this VBS script run ok in Firefox on your system?
http://wiki.imacros.net/send-macro-code.vbs
It works fine in my tests.

3. iimGetLastExtract(), iimGetLastExtract(0), iimGetLastExtract(1) always returns "OK (1)" if there is only one extract statement. If there are 2 extract statements, iimGetLastExtract(), iimGetLastExtract(0) returns the data but iimGetLastExtract(1) and iimGetLastExtract(2) returns "OK (1)" again.


Do you have a test macro for us?

4. With the iMacros browser (and IE also)
iimPlay("CODE: TAG POS=1 ....") followed by iimPlay("CODE: TAG POS=R1 ...") does not work. The anchor somehow was lost so the following relative TAG failed. The same sequence works fine if they are sent together.


Confirmed. But is there a reason not to send it together? We recommend that, also for performance reasons.

5. Performance seems to be quite slow: the Firefox javascript version is significantly faster than the remote control scripting version, executing the same sequence of iimPlay("CODE: TAG ...") statements. Lots of CPU cycles are consumed by the remote control version as compared to the javascript version.

The external interface has to send all commands to the browser, so that takes longer by design. On the plus side, it is robust against all kinds of browser crashes.

But it seems you are sending each command in a separate iimPlay command? We highly recommend to add them together and send them at once:

Code: Select all
MyMacroCode = "CODE:"
MyMacroCode = MyMacroCode+"VERSION BUILD=7200814" + vbNewLine
MyMacroCode = MyMacroCode+"URL GOTO=http://www.iopus.com" + vbNewLine
MyMacroCode = MyMacroCode+"URL GOTO=http://forum.iopus.com"

'Run the macro
iret = iim1.iimPlay(MyMacroCode)
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm

Re: How to automate Firefox with iimInit ("-fx")

by winxp4all on Tue Mar 08, 2011 2:31 pm

Thank you for your quick reply!
Here is a very simple VBS script you can try:

Code: Select all
Option Explicit
Dim iim1, iret, s
set iim1 = CreateObject ("imacros")
iret = iim1.iimInit("-fx")
s = "CODE: URL GOTO=http://getfirebug.com/firstrun#Firebug%201.6.2" + vbNewLine
s = s +  "TAG POS=1 TYPE=A ATTR=REL:bookmark EXTRACT=TXT" + vbNewLine
s = s +  "TAG POS=R1 TYPE=A ATTR=REL:bookmark EXTRACT=TXT"
iret = iim1.iimPlay(s)
MsgBox (iim1.iimGetLastExtract(1))
MsgBox (iim1.iimGetLastExtract(2))
MsgBox (iim1.iimGetLastExtract(0))
WScript.Quit(0)

I left the firefox browser open so you can see what is going on. What I got was extrac(1) = "OK (1)", extract(2) = null, extract(0) = both extracted fields correctly returned.

Now, close the Firefox browser and manually brings it up again. While Firefox is at it's default home page, run the VBS script again. On my laptop, a new Firefox browser did not open but the script seems to run identically to the first time when there was no open Firefox. Same issues with extraction. Very strange!

Here is another simple VBS script
Code: Select all
Option Explicit
Dim iim1, iret, s
set iim1 = CreateObject ("imacros")
iret = iim1.iimInit("-fx")
s = "CODE: URL GOTO=http://getfirebug.com/firstrun#Firebug%201.6.2" + vbNewLine
s = s +  "TAG POS=1 TYPE=A ATTR=REL:bookmark EXTRACT=TXT"
iret = iim1.iimPlay(s)
MsgBox (iim1.iimGetLastExtract(1))
s = "CODE: TAG POS=R1 TYPE=A ATTR=REL:bookmark EXTRACT=TXT"
iret = iim1.iimPlay(s)
MsgBox (iim1.iimGetLastExtract(1))
WScript.Quit(0)


Make sure there is NO running Firefox before you run this script. What I see is that the script has no problem with the relative TAG command even though the issues of extraction prevent the return of the data.

Now, change the script to call the iMacros browser instead. For me, the relative TAG command failed to find the second target. Notice that there is no problem with extraction in this case.

The reason I want to do this is to perform sequential extractions from within Python, checking the data returned and stop when I see "#EANF". I can pass POS=(1,2,3...) from Python but that is very inefficient. It's better to start with POS=1 and change to POS=R1 with subsequent commands. It seems to work in Firefox, but then I get nothing back so I am stuck either way!
winxp4all
 
Posts: 9
Joined: Tue Mar 08, 2011 1:09 am

Re: How to automate Firefox with iimInit ("-fx")

by winxp4all on Tue Mar 08, 2011 2:45 pm

Another issue that I found out with the Firefox Javascript interface. While I find this interface to be very solid and without any problem so far, a long-running javascript that performs many extractions from many different pages will eventually cause Firefox to crash. The reason seems to be "out of memory" from what I can see with the Task Manager performance tab. It looks like a case of memory leaks and memory fragmentation. I'd expect the same problem with running the scripting version. Have you heard of any such problem from users who run iMacros jobs 24x7? Can this be done?
winxp4all
 
Posts: 9
Joined: Tue Mar 08, 2011 1:09 am

Re: How to automate Firefox with iimInit ("-fx")

by Tech Support on Tue Mar 08, 2011 2:58 pm

Another issue that I found out with the Firefox Javascript interface. While I find this interface to be very solid and without any problem so far, a long-running javascript that performs many extractions from many different pages will eventually cause Firefox to crash. The reason seems to be "out of memory"...

This is correct. By design browsers are not intended for 24x7 operations and running them repeatedly for several days can lead to undesirable effects, such as increased memory consumption ("memory leaks"). So that is not an iMacros bug, but a general web browser issue.

Solution: See below :)

I'd expect the same problem with running the scripting version. Have you heard of any such problem from users who run iMacros jobs 24x7? Can this be done?

It can be done: The regular "outside" scripting interface avoids this issue completely, as it start a new browser instance all the time. Please see this FAQ for details: How can I set up a 24 hours a day, 7 days a week (non-stop) operation with iMacros?
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm

Re: How to automate Firefox with iimInit ("-fx")

by Tech Support on Tue Mar 08, 2011 3:03 pm

>The reason I want to do this is to perform sequential extractions from within Python

I still do not understand why you do want to extract the complete page at once?

That info is not required for fixing this bug, but I try to understand the use case. Can you post a link to a sample website? Maybe I can suggest a workaround.
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm

Re: How to automate Firefox with iimInit ("-fx")

by winxp4all on Tue Mar 08, 2011 3:40 pm

Your suggested method of running iMacros 24x7 works fine for repeating many small to medium size scripts. For a larger script that needs to extract many thousands pieces of data in one run, the job may have to be partitioned into segments that run sequentially with some way to resume from where the previous one was left off.

Here is the Python script that uses relative TAG to loop through a web page
Code: Select all
import win32com.client
w = win32com.client.Dispatch("imacros")
w.iimInit("-fx")
w.iimPlay("CODE: URL GOTO=http://getfirebug.com/firstrun#Firebug%201.6.2")
pos = '1'
while (1):
  w.iimPlay("CODE: TAG POS="+pos+" TYPE=A ATTR=REL:bookmark EXTRACT=TXT")
  pos = 'R1'
  s  = w.iimGetLastExtract(1)
  if s == "#EANF#": break
  print s

If you run this script with Firefox, the bad returned data will make it loop forever!
Run this script with iMacros browser and you will get only the first extracted item!
winxp4all
 
Posts: 9
Joined: Tue Mar 08, 2011 1:09 am

Re: How to automate Firefox with iimInit ("-fx")

by winxp4all on Thu Mar 10, 2011 2:41 pm

I have noticed that iMacros for iMacros browser runs quite a bit slower than iMacros for Firefox. It appears that the delay between the TAG EXTRACT commands is much longer on iMacros browser than on Firefox. I've checked the Play speed option and it is set to Fast. Is this by design or is there some setting that I am not aware of?
winxp4all
 
Posts: 9
Joined: Tue Mar 08, 2011 1:09 am

Re: How to automate Firefox with iimInit ("-fx")

by Tech Support on Thu Mar 10, 2011 3:15 pm

>Is this by design or is there some setting that something am not aware of?

This is a known issue, and we are currently working on it. In fact, the first tuned version was released a few days ago:

http://wiki.imacros.net/Test_Builds
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm

Re: How to automate Firefox with iimInit ("-fx")

by Tech Support on Thu Mar 10, 2011 3:21 pm

For a larger script that needs to extract many thousands pieces of data in one run, the job may have to be partitioned into segments that run sequentially with some way to resume from where the previous one was left off.


What kind of extraction project would this be? Frankly, I can not think of any task that can not be solved with our recommended setup. You are welcome to email the details to support AT iopus.com if you can not post it in a public forum.

On our real browser monitoring service we run 1000s of session each day with this approach. Of course, fully-automated and non-stop.

Mathias
User avatar
Tech Support
 
Posts: 5003
Joined: Tue Sep 20, 2005 12:25 pm

Re: How to automate Firefox with iimInit ("-fx")

by winxp4all on Thu Mar 10, 2011 9:01 pm

I've just finished two scripts that performed a few hundred extractions of both text and images as a test. The javascript version inside Firefox clearly showed memory leaks and the eventual crash of firefox. I am not sure who the culprit is. The scripting version using IE on the contrary showed quite stable memory utilization and finished without a crash. The downside is it took quite a bit longer to run the test with IE than with Firefox. I cannot test the scripting version with Firefox due to the issue of TAG EXTRACT mentioned above. One thumb up and two thumbs down for iMacros. Keep up the good work because this is a useful tool.
winxp4all
 
Posts: 9
Joined: Tue Mar 08, 2011 1:09 am

Re: How to automate Firefox with iimInit ("-fx")

by winxp4all on Sun Mar 13, 2011 8:58 pm

Your example of how to use iMacros scripting version with Java does not show how to check for return codes from the scripting commands. for example:
Code: Select all
iim.invoke("iimPlay","CODE: ...)

how does one check the return code of iimPlay?

Another question: how does one invoke iimInit("-ie",false) in a Java program, using JACOB?
Last edited by winxp4all on Sun Mar 13, 2011 9:35 pm, edited 1 time in total.
winxp4all
 
Posts: 9
Joined: Tue Mar 08, 2011 1:09 am

Re: How to automate Firefox with iimInit ("-fx")

by winxp4all on Sun Mar 13, 2011 9:14 pm

I tried this and it seems to work:
Code: Select all
int rc = Integer.parseInt(iim.invoke("iimPlay","CODE: ...").toString())

Is there a better way than this convoluted expression?

I still cannot figure out how to invoke iimInit("-ie",false) using JACOB!
winxp4all
 
Posts: 9
Joined: Tue Mar 08, 2011 1:09 am

PreviousNext

Return to Scripting and Command Line Interface

Who is online

Users browsing this forum: No registered users and 1 guest

-->