PROXY multiple BYPASS with *keywords*

Support for iMacros. The iMacros software is the unique solution for automating every activity inside a web browser, for data extraction and web testing.
Forum rules
Before asking a question or reporting an issue:
1. Please review the list of FAQ's.
2. Use the search box (at the top of each forum page) to see if a similar problem or question has already been addressed.
3. Try searching the iMacros Wiki - it contains the complete iMacros reference as well as plenty of samples and tutorials.
4. We can respond much faster to your posts if you include the following information: CLICK HERE FOR IMPORTANT INFORMATION TO INCLUDE IN YOUR POST
Post Reply
User avatar
cybmatrix
Posts: 38
Joined: Tue Dec 08, 2009 8:33 pm

PROXY multiple BYPASS with *keywords*

Post by cybmatrix » Thu Sep 05, 2019 1:52 pm

Hi there.
I'm wondering about how to manage multiple bypass keywords for PROXY setting in macro.

PROXY ADDRESS=XXX.XXX.XXX.XXX:8080 BYPASS=*imacros*

This is a single keyword exclusion.
And if I want to use many keywords ?

thanks. :mrgreen:

-----------

Windows 7 PRO (64 bit)
Waterfox 51.0.1 (64 bit)
iMacros for FF 8.9.7
User avatar
thecoder2012
Posts: 367
Joined: Sat Aug 15, 2015 5:14 pm
Location: Internet
Contact:

Re: PROXY multiple BYPASS with *keywords*

Post by thecoder2012 » Thu Sep 05, 2019 2:10 pm

cybmatrix wrote:
Thu Sep 05, 2019 1:52 pm
I'm wondering about how to manage multiple bypass keywords for PROXY setting in macro.
PROXY ADDRESS=XXX.XXX.XXX.XXX:8080 BYPASS=*imacros*

This is a single keyword exclusion.
And if I want to use many keywords ?
See about the proxy,a big error in wiki

Example:

Code: Select all

PROXY ADDRESS=127.0.0.1:3128 BYPASS="iplocation.com, whatismyipaddress.com"
URL GOTO=https://iplocation.com/
PAUSE
URL GOTO=https://whatismyipaddress.com/
PAUSE
But ".domain.tld" is only for urls with subdomains. Without subdomain you can use "domain.tld" for your bypass.
Tested with Waterfox 56, iMacros 8.9.7, Windows 8.1 (x64)
Join 9kw.eu Captcha Service now and let your iMacros continue downloads and scripts while you sleep. - Custom iMacros? Contact me! :idea:
User avatar
cybmatrix
Posts: 38
Joined: Tue Dec 08, 2009 8:33 pm

Re: PROXY multiple BYPASS with *keywords*

Post by cybmatrix » Thu Sep 05, 2019 4:37 pm

And how to pass also USERNAME and PASS for the proxy login ?

user:pass@000.000.000.000:8080 is not supported by iMacros.


thanks
User avatar
thecoder2012
Posts: 367
Joined: Sat Aug 15, 2015 5:14 pm
Location: Internet
Contact:

Re: PROXY multiple BYPASS with *keywords*

Post by thecoder2012 » Sun Sep 08, 2019 12:11 pm

cybmatrix wrote:
Thu Sep 05, 2019 4:37 pm
And how to pass also USERNAME and PASS for the proxy login ?
user:pass@000.000.000.000:8080 is not supported by iMacros.
Not possible without JavaScript in my eyes or foxyproxy. You can use addons (e.g. foxyproxy), third party proxy software (e.g. ccproxy => Options => Advanced => Cascading) or javascript with your imacros 8.9.7
Last edited by thecoder2012 on Wed Sep 11, 2019 10:56 pm, edited 1 time in total.
Join 9kw.eu Captcha Service now and let your iMacros continue downloads and scripts while you sleep. - Custom iMacros? Contact me! :idea:
User avatar
cybmatrix
Posts: 38
Joined: Tue Dec 08, 2009 8:33 pm

Re: PROXY multiple BYPASS with *keywords*

Post by cybmatrix » Mon Sep 09, 2019 3:11 pm

Will try with Foxyproxy.
Thanks.
User avatar
cybmatrix
Posts: 38
Joined: Tue Dec 08, 2009 8:33 pm

Re: PROXY multiple BYPASS with *keywords*

Post by cybmatrix » Mon Sep 09, 2019 3:36 pm

And what about the javascript solution instead ?
User avatar
thecoder2012
Posts: 367
Joined: Sat Aug 15, 2015 5:14 pm
Location: Internet
Contact:

Re: PROXY multiple BYPASS with *keywords*

Post by thecoder2012 » Thu Sep 12, 2019 10:51 am

cybmatrix wrote:
Mon Sep 09, 2019 3:36 pm
And what about the javascript solution instead ?
Foxyproxy source code. This feature currently can only be used from an XUL extension using the XPCOM API (e.g. waterfox 56, PaleMoon/Basilisk, old firefox). Firefox 45 and higher supports SOCKS proxies with authentication. Before that, SOCKS is supported but not with authentication.
I think it's usable in imacros 8.9.7 with javascript.

1. HTTP/HTTPS Proxy (with authentication)
Easier than socks5 proxy in my eyes like observer with "http-on-modify-request" with username and password. Set proxy and header modification with "Proxy-Authorizationn" + "Basic USER:PASS". USER:PASS is in Base64 required (e.g. in the source code Proxy Authentication #1 #2 => chrome => content => overlay.js).
See https://developer.mozilla.org/en-US/doc ... horization and https://developer.mozilla.org/en-US/doc ... entication
Or you can try https://stackoverflow.com/questions/637 ... -extension but I think it's only for http authentication.

2. Socks4/5 proxy (with authentication)
Solution is in Foxyproxy. I have not enough time to research it in the addon source code.

It's tricky (e.g. set isHttps to false, isSocks to true) but helpful are lines like (foxyproxy 4.6.5 => components => proxy.js):
var proxyService = CC["@mozilla.org/network/protocol-proxy-service;1"].getService(CI.nsIProtocolProxyService),
proxyService.newProxyInfoWithAuth(this._socksversion == "5" ? "socks" : "socks4", this._host, this._port, this.username, this.password, this.owner._proxyDNS ? CI.nsIProxyInfo.TRANSPARENT_PROXY_RESOLVES_HOST : 0, 0, null);
See https://developer.mozilla.org/en-US/doc ... oxyService

source/toolkit/components/extensions/ProxyScriptContext.jsm:
newProxyInfoWithAuth:
type,
host,
port,
username,
password,
proxyAuthorizationHeader,
connectionIsolationKey,
proxyDNS ? TRANSPARENT_PROXY_RESOLVES_HOST : 0,
failoverTimeout ? failoverTimeout : PROXY_TIMEOUT_SEC,
failoverProxy

Untested (except foxyproxy?) and no documentation in my eyes.

3. Use foyproxy in your addons (e.g.imacros 8.9.7)
Easiest way is the protocol handler with your addressbar (url goto): https://getfoxyproxy.org/developers/proxyprotocol/ (with geoip check page under https://getfoxyproxy.org/geoip/)
Release notes: https://help.getfoxyproxy.org/index.php ... r-firefox/ ("support of proxy authentication credentials" since 2013/v4.2+)

Example without JavaScript and Foxyproxy (v4.6.5):
proxy:host=foo.tld&port=999&enabled=true&proxyDNS=false&password=foobar&username=foouser&foxyProxyMode=this&mode=manual&socksversion=5&isSocks=true

My test script with pure IIM (with ccproxy free as proxy software with socks5 under Windows):

Code: Select all

TAB T=1
URL GOTO=proxy:host=foo.tld&port=999&enabled=true&proxyDNS=false&username=foouser&password=foobar&foxyProxyMode=this&mode=manual&socksversion=5&isSocks=true
URL GOTO=https://getfoxyproxy.org/geoip/
URL GOTO=proxy:host=foo.tld&port=999&enabled=true&proxyDNS=false&username=foouser&password=foobar&foxyProxyMode=this&mode=manual&socksversion=5&isSocks=true&action=delete
It works very fast and very good in my tests. No delays in my tests.

But it's tricky with iMacros without Button "Allow" for every foxyproxy api call.

Download the original XPI File (v4.6.5), rename xpi to zip and use winrar (update the zip file, don't create a new zip file!).
1. File components/common.js
Search:

Code: Select all

notify : function(msg, ar, buttons, priority, callback, getNotWithVal, callbackArgs) {
Replace it with this code:

Code: Select all

notify : function(msg, ar, buttons, priority, callback, getNotWithVal, callbackArgs) {
          return callback(callbackArgs);
2. File components/proxy.js
Search:

Code: Select all

if (this.username && this.password && this.fp.isGecko45 && this._socksversion == "5") {
Replace it:

Code: Select all

if (this.username && this.password && this._socksversion == "5") {
Rename it back from zip to xpi and try it.
NOTE: If you have foxyproxy already installed then deinstall it OR delete the folder foxyproxy* under "extensions" in your profile directory (and restart your browser)!

Or you can use the xpi file in the attachment (4 rar archive, 3 in this post and 1 in the next post). Tested with Basilisk Browser 2019.09.03 (64-bit), Win8.1 (x64) and iMacros v8.9.7
Attachments
foxyproxy-standard-4.6.5_imacros-mod.part3.rar
(250 KiB) Downloaded 12 times
foxyproxy-standard-4.6.5_imacros-mod.part2.rar
(250 KiB) Downloaded 10 times
foxyproxy-standard-4.6.5_imacros-mod.part1.rar
(250 KiB) Downloaded 11 times
Last edited by thecoder2012 on Thu Sep 12, 2019 11:06 am, edited 3 times in total.
Join 9kw.eu Captcha Service now and let your iMacros continue downloads and scripts while you sleep. - Custom iMacros? Contact me! :idea:
User avatar
thecoder2012
Posts: 367
Joined: Sat Aug 15, 2015 5:14 pm
Location: Internet
Contact:

Re: PROXY multiple BYPASS with *keywords*

Post by thecoder2012 » Thu Sep 12, 2019 10:54 am

part4 rar (forum is limited in this case)

First mod as proof of concept for new ideas without modification imacros xpi file. :D
Attachments
foxyproxy-standard-4.6.5_imacros-mod.part4.rar
(143.9 KiB) Downloaded 13 times
Join 9kw.eu Captcha Service now and let your iMacros continue downloads and scripts while you sleep. - Custom iMacros? Contact me! :idea:
chivracq
Posts: 8781
Joined: Sat Apr 13, 2013 1:07 pm
Location: Amsterdam (NL)

Re: PROXY multiple BYPASS with *keywords*

Post by chivracq » Fri Sep 13, 2019 12:02 am

thecoder2012 wrote:
Thu Sep 12, 2019 10:54 am
First mod as proof of concept for new ideas without modification imacros xpi file. :D

Nice Work, @thecoder2012..., fouff...!, I'm impressed, ah-ah...!! :wink:

'7-Zip' is better to use than 'WinRar' btw, I think... (for editing), 'WinRar' used to not even be able to edit '.zip'/'.rar'/'.xpi' Files, must be recent..., after it came "in the News" about 6 or 12 months ago for some "Design/Security-Flaws" that had been pledging 'WinRar' for 17 years if I remember correctly...
I think with '7-Zip', you don't even need to change the Extension anymore...

But hum, yep-yep, exactly what I thought "without modification imacros xpi file", would be "nice" to have that Func directly integrated in iMacros (for FF), won't happen from @Dev, as even if they did it, that would be only be for some v10.0.x next Version for FF, no chance they will ever add it to v8.9.7 for FF... :twisted:

Maybe "Time" to revive this older "Project", ah-ah...!:
- Re: Extend Imacros with new commands?
What...!?, Thread is already from 6 years ago, oops-oops...!! :shock: Hum, I was already full of "Good Ideas" then, ah-ah...! 8) , hum, and already customizing the '.xpi' Add-on for FF apparently, re-oops...! :oops: Hum, but yep of course, I was doing it already "long" before I joined the Forum in May 2013... Shit..., 6 years already...! :o

>>>
Post review
This post has been altered by another user while you were editing it. You may wish to review the current version of this post and adjust your edits.
... While trying to save some Edit...
=> Yep, don't worry, that was me already the "other User"... :P
- (F)CI(M) = (Full) Config Info (Missing): iMacros + Browser + OS (+ all 3 Versions + 'Free'/'PE').
- I don't even read the Qt if that (required) Info is not mentioned...!
- Script & URL help a lot for more "educated" Help...
User avatar
thecoder2012
Posts: 367
Joined: Sat Aug 15, 2015 5:14 pm
Location: Internet
Contact:

Re: PROXY multiple BYPASS with *keywords*

Post by thecoder2012 » Fri Sep 13, 2019 8:05 am

chivracq wrote:
Fri Sep 13, 2019 12:02 am
But hum, yep-yep, exactly what I thought "without modification imacros xpi file", would be "nice" to have that Func directly integrated in iMacros (for FF), won't happen from @Dev, as even if they did it, that would be only be for some v10.0.x next Version for FF, no chance they will ever add it to v8.9.7 for FF... :twisted:
No chance? Not the usual way, yes. But we have more ways. :)
Old addon system is very powerful.
chivracq wrote:
Fri Sep 13, 2019 12:02 am
Maybe "Time" to revive this older "Project", ah-ah...!:
No time and socks proxy function (with gui) was required for my customer scripts.
chivracq wrote:
Fri Sep 13, 2019 12:02 am
What...!?, Thread is already from 6 years ago, oops-oops...!! :shock: Hum, I was already full of "Good Ideas" then, ah-ah...! 8) , hum, and already customizing the '.xpi' Add-on for FF apparently, re-oops...! :oops: Hum, but yep of course, I was doing it already "long" before I joined the Forum in May 2013... Shit..., 6 years already...! :o
And which solutions are available since 6 years? :wink:
Join 9kw.eu Captcha Service now and let your iMacros continue downloads and scripts while you sleep. - Custom iMacros? Contact me! :idea:
User avatar
cybmatrix
Posts: 38
Joined: Tue Dec 08, 2009 8:33 pm

Re: PROXY multiple BYPASS with *keywords*

Post by cybmatrix » Tue Sep 17, 2019 2:14 pm

Great great great work my friend.
I'm impressed.

Thank you for your time.
Post Reply