How to automate Firefox with iimInit ("-fx")

Discussions and Tech Support related to automating the iMacros Browser or Internet Explorer from any scripting and programming language, such as VBS (WSH), VBA, VB, Perl, Delphi, C# or C++.
Forum rules
iMacros EOL - Attention!

The renewal maintenance has officially ended for Progress iMacros effective November 20, 2023 and all versions of iMacros are now considered EOL (End-of-Life). The iMacros products will no longer be supported by Progress (aside from customer license issues), and these forums will also no longer be moderated from the Progress side.

Thank you again for your business and support.

Sincerely,
The Progress Team

Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Tom, Tech Support
Posts: 3834
Joined: Mon May 31, 2010 4:59 pm

Re: How to automate Firefox with iimInit ("-fx")

Post by Tom, Tech Support » Tue Feb 22, 2011 7:13 pm

Hello echan00,

Would you please describe exactly what problem you are experiencing. This thread (and the other thread you previously posted too) pertain primarily to using the iMacros Scripting Interface to automate Firefox.

There shouldn't be any problems with using the built-in Javascript scripting interface in iMacros for Firefox.
Regards,

Tom, iMacros Support
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Re: How to automate Firefox with iimInit ("-fx")

Post by Tech Support » Wed Mar 02, 2011 4:22 pm

Update: iMacros V7.22 and iMacros for Firefox 7.1.1.5 solve the issue reported by mjayson.
winxp4all
Posts: 9
Joined: Tue Mar 08, 2011 8:09 am

Re: How to automate Firefox with iimInit ("-fx")

Post by winxp4all » Tue Mar 08, 2011 9:26 am

Windows version = Windows 7 Ultimate, 32-bit, Service Pack 1
iMacros scripting version = 7.22.1254
Firefox version = 3.6.15
Firefox iMacros plug-in version = 7.1.1.5
Python version 3.2 with win32com.client build 216

There are many serious problems:

1. iimInit("-fx",False) does not work, always starts a new Firefox session.
2. iimInit("-fx") followed by iimPlay("CODE: URL GOTO=...") does not work. The iimPlay() is sent before the browser comes up and therefore gets lost.
3. iimGetLastExtract(), iimGetLastExtract(0), iimGetLastExtract(1) always returns "OK (1)" if there is only one extract statement. If there are 2 extract statements, iimGetLastExtract(), iimGetLastExtract(0) returns the data but iimGetLastExtract(1) and iimGetLastExtract(2) returns "OK (1)" again.
4. With the iMacros browser (and IE also)
iimPlay("CODE: TAG POS=1 ....") followed by iimPlay("CODE: TAG POS=R1 ...") does not work. The anchor somehow was lost so the following relative TAG failed. The same sequence works fine if they are sent together.
iimPlay("CODE: TAG POS=1 ....") followed by iimPlay("CODE: TAG POS=R1 ...") works fine with Firefox so relative tagging with iimPlay() must be OK. It just does not work with iMacros browser and IE.
5. Performance seems to be quite slow: the Firefox javascript version is significantly faster than the remote control scripting version, executing the same sequence of iimPlay("CODE: TAG ...") statements. Lots of CPU cycles are consumed by the remote control version as compared to the javascript version.

It looks like the current iMacros scripting version is far from stable for production work.
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Re: How to automate Firefox with iimInit ("-fx")

Post by Tech Support » Tue Mar 08, 2011 12:03 pm

Thanks you for the system details. They look all ok to me.
1. iimInit("-fx",False) does not work, always starts a new Firefox session.
Yes, FALSE is not supported for Firefox. This parameter usually is not required for unattended automation. If you have a special use case that would require please let us know.
2. iimInit("-fx") followed by iimPlay("CODE: URL GOTO=...") does not work. The iimPlay() is sent before the browser comes up and therefore gets lost.
That works fine in our test. As a test, does this VBS script run ok in Firefox on your system?
http://wiki.imacros.net/send-macro-code.vbs
It works fine in my tests.
3. iimGetLastExtract(), iimGetLastExtract(0), iimGetLastExtract(1) always returns "OK (1)" if there is only one extract statement. If there are 2 extract statements, iimGetLastExtract(), iimGetLastExtract(0) returns the data but iimGetLastExtract(1) and iimGetLastExtract(2) returns "OK (1)" again.
Do you have a test macro for us?
4. With the iMacros browser (and IE also)
iimPlay("CODE: TAG POS=1 ....") followed by iimPlay("CODE: TAG POS=R1 ...") does not work. The anchor somehow was lost so the following relative TAG failed. The same sequence works fine if they are sent together.
Confirmed. But is there a reason not to send it together? We recommend that, also for performance reasons.
5. Performance seems to be quite slow: the Firefox javascript version is significantly faster than the remote control scripting version, executing the same sequence of iimPlay("CODE: TAG ...") statements. Lots of CPU cycles are consumed by the remote control version as compared to the javascript version.
The external interface has to send all commands to the browser, so that takes longer by design. On the plus side, it is robust against all kinds of browser crashes.

But it seems you are sending each command in a separate iimPlay command? We highly recommend to add them together and send them at once:

Code: Select all

MyMacroCode = "CODE:"
MyMacroCode = MyMacroCode+"VERSION BUILD=7200814" + vbNewLine
MyMacroCode = MyMacroCode+"URL GOTO=http://www.iopus.com" + vbNewLine
MyMacroCode = MyMacroCode+"URL GOTO=http://forum.imacros.net"

'Run the macro
iret = iim1.iimPlay(MyMacroCode)
winxp4all
Posts: 9
Joined: Tue Mar 08, 2011 8:09 am

Re: How to automate Firefox with iimInit ("-fx")

Post by winxp4all » Tue Mar 08, 2011 9:31 pm

Thank you for your quick reply!
Here is a very simple VBS script you can try:

Code: Select all

Option Explicit
Dim iim1, iret, s
set iim1 = CreateObject ("imacros")
iret = iim1.iimInit("-fx")
s = "CODE: URL GOTO=http://getfirebug.com/firstrun#Firebug%201.6.2" + vbNewLine
s = s +  "TAG POS=1 TYPE=A ATTR=REL:bookmark EXTRACT=TXT" + vbNewLine
s = s +  "TAG POS=R1 TYPE=A ATTR=REL:bookmark EXTRACT=TXT"
iret = iim1.iimPlay(s)
MsgBox (iim1.iimGetLastExtract(1))
MsgBox (iim1.iimGetLastExtract(2))
MsgBox (iim1.iimGetLastExtract(0))
WScript.Quit(0)
I left the firefox browser open so you can see what is going on. What I got was extrac(1) = "OK (1)", extract(2) = null, extract(0) = both extracted fields correctly returned.

Now, close the Firefox browser and manually brings it up again. While Firefox is at it's default home page, run the VBS script again. On my laptop, a new Firefox browser did not open but the script seems to run identically to the first time when there was no open Firefox. Same issues with extraction. Very strange!

Here is another simple VBS script

Code: Select all

Option Explicit
Dim iim1, iret, s
set iim1 = CreateObject ("imacros")
iret = iim1.iimInit("-fx")
s = "CODE: URL GOTO=http://getfirebug.com/firstrun#Firebug%201.6.2" + vbNewLine
s = s +  "TAG POS=1 TYPE=A ATTR=REL:bookmark EXTRACT=TXT"
iret = iim1.iimPlay(s)
MsgBox (iim1.iimGetLastExtract(1))
s = "CODE: TAG POS=R1 TYPE=A ATTR=REL:bookmark EXTRACT=TXT"
iret = iim1.iimPlay(s)
MsgBox (iim1.iimGetLastExtract(1))
WScript.Quit(0)
Make sure there is NO running Firefox before you run this script. What I see is that the script has no problem with the relative TAG command even though the issues of extraction prevent the return of the data.

Now, change the script to call the iMacros browser instead. For me, the relative TAG command failed to find the second target. Notice that there is no problem with extraction in this case.

The reason I want to do this is to perform sequential extractions from within Python, checking the data returned and stop when I see "#EANF". I can pass POS=(1,2,3...) from Python but that is very inefficient. It's better to start with POS=1 and change to POS=R1 with subsequent commands. It seems to work in Firefox, but then I get nothing back so I am stuck either way!
winxp4all
Posts: 9
Joined: Tue Mar 08, 2011 8:09 am

Re: How to automate Firefox with iimInit ("-fx")

Post by winxp4all » Tue Mar 08, 2011 9:45 pm

Another issue that I found out with the Firefox Javascript interface. While I find this interface to be very solid and without any problem so far, a long-running javascript that performs many extractions from many different pages will eventually cause Firefox to crash. The reason seems to be "out of memory" from what I can see with the Task Manager performance tab. It looks like a case of memory leaks and memory fragmentation. I'd expect the same problem with running the scripting version. Have you heard of any such problem from users who run iMacros jobs 24x7? Can this be done?
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Re: How to automate Firefox with iimInit ("-fx")

Post by Tech Support » Tue Mar 08, 2011 9:58 pm

Another issue that I found out with the Firefox Javascript interface. While I find this interface to be very solid and without any problem so far, a long-running javascript that performs many extractions from many different pages will eventually cause Firefox to crash. The reason seems to be "out of memory"...
This is correct. By design browsers are not intended for 24x7 operations and running them repeatedly for several days can lead to undesirable effects, such as increased memory consumption ("memory leaks"). So that is not an iMacros bug, but a general web browser issue.

Solution: See below :)
I'd expect the same problem with running the scripting version. Have you heard of any such problem from users who run iMacros jobs 24x7? Can this be done?
It can be done: The regular "outside" scripting interface avoids this issue completely, as it start a new browser instance all the time. Please see this FAQ for details: How can I set up a 24 hours a day, 7 days a week (non-stop) operation with iMacros?
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Re: How to automate Firefox with iimInit ("-fx")

Post by Tech Support » Tue Mar 08, 2011 10:03 pm

>The reason I want to do this is to perform sequential extractions from within Python

I still do not understand why you do want to extract the complete page at once?

That info is not required for fixing this bug, but I try to understand the use case. Can you post a link to a sample website? Maybe I can suggest a workaround.
winxp4all
Posts: 9
Joined: Tue Mar 08, 2011 8:09 am

Re: How to automate Firefox with iimInit ("-fx")

Post by winxp4all » Tue Mar 08, 2011 10:40 pm

Your suggested method of running iMacros 24x7 works fine for repeating many small to medium size scripts. For a larger script that needs to extract many thousands pieces of data in one run, the job may have to be partitioned into segments that run sequentially with some way to resume from where the previous one was left off.

Here is the Python script that uses relative TAG to loop through a web page

Code: Select all

import win32com.client
w = win32com.client.Dispatch("imacros")
w.iimInit("-fx")
w.iimPlay("CODE: URL GOTO=http://getfirebug.com/firstrun#Firebug%201.6.2")
pos = '1'
while (1):
  w.iimPlay("CODE: TAG POS="+pos+" TYPE=A ATTR=REL:bookmark EXTRACT=TXT")
  pos = 'R1'
  s  = w.iimGetLastExtract(1)
  if s == "#EANF#": break
  print s
If you run this script with Firefox, the bad returned data will make it loop forever!
Run this script with iMacros browser and you will get only the first extracted item!
winxp4all
Posts: 9
Joined: Tue Mar 08, 2011 8:09 am

Re: How to automate Firefox with iimInit ("-fx")

Post by winxp4all » Thu Mar 10, 2011 9:41 pm

I have noticed that iMacros for iMacros browser runs quite a bit slower than iMacros for Firefox. It appears that the delay between the TAG EXTRACT commands is much longer on iMacros browser than on Firefox. I've checked the Play speed option and it is set to Fast. Is this by design or is there some setting that I am not aware of?
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Re: How to automate Firefox with iimInit ("-fx")

Post by Tech Support » Thu Mar 10, 2011 10:15 pm

>Is this by design or is there some setting that something am not aware of?

This is a known issue, and we are currently working on it. In fact, the first tuned version was released a few days ago:

http://wiki.imacros.net/Test_Builds
User avatar
Tech Support
Posts: 4948
Joined: Tue Sep 20, 2005 7:25 pm
Contact:

Re: How to automate Firefox with iimInit ("-fx")

Post by Tech Support » Thu Mar 10, 2011 10:21 pm

For a larger script that needs to extract many thousands pieces of data in one run, the job may have to be partitioned into segments that run sequentially with some way to resume from where the previous one was left off.
What kind of extraction project would this be? Frankly, I can not think of any task that can not be solved with our recommended setup. You are welcome to email the details to support AT iopus.com if you can not post it in a public forum.

On our real browser monitoring service we run 1000s of session each day with this approach. Of course, fully-automated and non-stop.

Mathias
winxp4all
Posts: 9
Joined: Tue Mar 08, 2011 8:09 am

Re: How to automate Firefox with iimInit ("-fx")

Post by winxp4all » Fri Mar 11, 2011 4:01 am

I've just finished two scripts that performed a few hundred extractions of both text and images as a test. The javascript version inside Firefox clearly showed memory leaks and the eventual crash of firefox. I am not sure who the culprit is. The scripting version using IE on the contrary showed quite stable memory utilization and finished without a crash. The downside is it took quite a bit longer to run the test with IE than with Firefox. I cannot test the scripting version with Firefox due to the issue of TAG EXTRACT mentioned above. One thumb up and two thumbs down for iMacros. Keep up the good work because this is a useful tool.
winxp4all
Posts: 9
Joined: Tue Mar 08, 2011 8:09 am

Re: How to automate Firefox with iimInit ("-fx")

Post by winxp4all » Mon Mar 14, 2011 3:58 am

Your example of how to use iMacros scripting version with Java does not show how to check for return codes from the scripting commands. for example:

Code: Select all

iim.invoke("iimPlay","CODE: ...)
how does one check the return code of iimPlay?

Another question: how does one invoke iimInit("-ie",false) in a Java program, using JACOB?
Last edited by winxp4all on Mon Mar 14, 2011 4:35 am, edited 1 time in total.
winxp4all
Posts: 9
Joined: Tue Mar 08, 2011 8:09 am

Re: How to automate Firefox with iimInit ("-fx")

Post by winxp4all » Mon Mar 14, 2011 4:14 am

I tried this and it seems to work:

Code: Select all

int rc = Integer.parseInt(iim.invoke("iimPlay","CODE: ...").toString())
Is there a better way than this convoluted expression?

I still cannot figure out how to invoke iimInit("-ie",false) using JACOB!
Post Reply