Skip to content

Instantly share code, notes, and snippets.

@Zamana
Last active March 20, 2023 12:36
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save Zamana/68060da614909ed3531acbda6154f67d to your computer and use it in GitHub Desktop.
Save Zamana/68060da614909ed3531acbda6154f67d to your computer and use it in GitHub Desktop.

Installing FlareSolverr on FreeBSD or FreeNAS/TrueNAS jail

FlareSolverr is a proxy server to bypass Cloudflare protection.

I found it while looking for a way to fix a problem: my Jackett installation one day started to give errors when trying to access 1337x. You got the point...

Here are the instructions in order to get FlareSolverr configured and running at boot in a FreeBSD system or in a FreeNAS/TrueNAS jail (which is my case, by the way).

This works only for version 2.1.0. For version 2.2.0, I was unable to make it work. I'm sorry.

This was made in a TrueNAS Core 12.0-U7 which uses 12.2-RELEASE-p11 as base.

All instructions consider that you will be using the root user to execute the commands and to run FlareSolver. If you want or need to use a regular user, adapt the instructions accordingly.

I'll not enter in details about creation of jails, but here is the main screen of my jail creation form for your reference:

image

So, after creating a simple jail (clone jail, vnet interface), follow these instructions:

1. Check if you need to set your date/time

In my case it is necessary:

rm /etc/localtime
ln -s /usr/share/zoneinfo/America/Sao_Paulo /etc/localtime

2. Update and upgrade your packages

Before anything else, make sure that your packages are updated and upgraded:

pkg update
pkg upgrade

3. Get FlareSolverr

Here you have the option to use Git or download the sources directly. It doesn't matter what you choose, the final product will be the same.

The advantage of the Git way is that it is easier to keep FlareSolverr updated.

So, follow 3.1 or 3.2, as you wish.

3.1 Using Git

Install the git-tiny package:

pkg install git-tiny

Then go to /usr/local/share folder and clone the repository:

cd /usr/local/share
git clone https://github.com/FlareSolverr/FlareSolverr.git

3.2 Using Zip

Go to the FlareSolverr Github page, click on Code, then right-click on "Download ZIP" and then choose "Copy link address":

image

Then execute the fetch command to download the zip file, by pasting the copied address in the command line, and then run the unzip command to extract it:

fetch https://github.com/FlareSolverr/FlareSolverr/archive/refs/heads/master.zip
unzip master.zip

Now move the FlareSolverr-master folder to /usr/local/share changing the folder's name to just FlareSolverr:

mv FlareSolverr-master /usr/local/share/FlareSolverr

4. Installing the necessary packages

Once you have the FlareSolverr source code downloaded through Git or Zip in /usr/local/share/FlareSolverr, the next step is to download the necessary packages.

Install these packages and all their required dependencies:

pkg install npm-node14 chromium

Around 170 packages should be installed.

5. Installing FlareSolverr

Notes:

  1. From now on, everything should be executed inside the FlareSolverr installation folder, that is /usr/local/share/FlareSolverr

  2. The command execution order is important!

  3. setenv is a CShell command; adapt if you're using another shell

  4. I confess that I don't understand yet why the double installation of puppeteer is necessary; I guess that this is a bug or something. I'll update this guide when I find the reason (I hope...)

Execute the following commands to install FlareSolverr:

cd /usr/local/share/FlareSolverr
setenv PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
setenv PUPPETEER_EXECUTABLE_PATH /usr/local/bin/chrome
npm install
node node_modules/puppeteer/install.js
npm install puppeteer@1.2.0
npm install puppeteer
npm run build

6. Testing

In order to test if FlareSolverr is configured and can run, execute:

cd /usr/local/share/FlareSolverr
npm start

If everything is fine, you should see something like this:

root@flaresolverr:/usr/local/share/FlareSolverr # npm start

> flaresolverr@2.1.0 start /usr/local/share/FlareSolverr
> node ./dist/server.js

2022-01-01T20:32:20-03:00 INFO FlareSolverr v2.1.0
2022-01-01T20:32:20-03:00 INFO Testing web browser installation...
2022-01-01T20:32:21-03:00 INFO FlareSolverr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/94.0.4606.81 Safari/537.36
2022-01-01T20:32:21-03:00 INFO Test successful
2022-01-01T20:32:21-03:00 INFO Listening on http://flaresolverr:8191

This means that FlareSolverr is running and listening on port 8191 in the localhost interface (which is useless in some cases, but we will fix this in no time).

Congratulations!

If not... get back and check the steps above.

7. Run at system boot/jail start

In order to have FlareSolverr running at system boot or jail start, create a file named /usr/local/etc/rc.d/flaresolverr with the following content:

#!/bin/sh

#
# Author: C. R. Zamana (czamana at gmail dot com)
#
# PROVIDE: flaresolverr
# REQUIRE: networking
# KEYWORD:

. /etc/rc.subr

name="flaresolverr"
rcvar="${name}_enable"
load_rc_config ${name}

: ${flaresolverr_enable:="NO"}

pidfile="/var/run/flaresolverr.pid"

start_precmd="flaresolverr_precmd"

PATH=$PATH:/usr/local/bin

flaresolverr_precmd() {
        cd /usr/local/share/FlareSolverr
        export PUPPETEER_EXECUTABLE_PATH=/usr/local/bin/chrome
        export HOST=0.0.0.0
}

command="/usr/sbin/daemon"
command_args="-P ${pidfile} /usr/local/bin/npm start > /dev/null"

run_rc_command "$1"

Give it the execute permissions:

chmod +x /usr/local/etc/rc.d/flaresolverr

And make it run at sytem boot/jail start:

sysrc flaresolverr_enable=YES

And now you are ready to control the service with:

service flaresolverr start | stop | status

Note that the script sets the variable HOST as 0.0.0.0 which will make FlareSolverr to listen in all interfaces, which is what you want if you are using it in a system and acessing it from another. Feel free to change this according with your needs.

Beyond service flaresolverr status, another way to check that the process is really running and listening at the port 8191, is:

netstat -an | grep 8191

8. FlareSolverr and Jackett: a use case

Now that you have FlareSolverr alive and kicking, you can use it in your Jackett installation, like this:

image

Happy FlareSolverring ;-)

@Cesar-Constant
Copy link

hello, flaresolverr works perfectly (i believe...) "{"msg":"FlareSolverr is ready!","version":"v2.1.0",bla bla bla" but each time jackett returns this error :
Exception (yggcookie): FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Error: Unable to process browser request. Error: Maximum timeout reached. maxTimeout=55000 (ms): FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Error: Unable to process browser request. Error: Maximum timeout reached. maxTimeout=55000 (ms)
Any idea?

@Zamana
Copy link
Author

Zamana commented Jan 17, 2022

That's a question that you need to ask to the FlareSolverr development team directly:

https://github.com/FlareSolverr/FlareSolverr/issues

@NasKar2
Copy link

NasKar2 commented Jan 31, 2022

New version of flaresolverr 2.2.0 gives errors : See below

root@flaresolverr2:/usr/local/share/FlareSolverr # npm run build

flaresolverr@2.2.0 build
tsc

src/controllers/v1.ts:2:24 - error TS2307: Cannot find module 'devtools-protocol' or its corresponding type declarations.

2 import {Protocol} from "devtools-protocol";
~~~~~~~~~~~~~~~~~~~

src/providers/cloudflare.ts:1:15 - error TS2305: Module '"puppeteer"' has no exported member 'HTTPResponse'.

1 import {Page, HTTPResponse} from 'puppeteer'
~~~~~~~~~~~~

src/services/sessions.ts:4:24 - error TS2307: Cannot find module 'devtools-protocol' or its corresponding type declarations.

4 import {Protocol} from "devtools-protocol";
~~~~~~~~~~~~~~~~~~~

src/services/solver.ts:1:15 - error TS2305: Module '"puppeteer"' has no exported member 'HTTPResponse'.

1 import {Page, HTTPResponse} from 'puppeteer'

@zajc3w
Copy link

zajc3w commented Feb 5, 2022

@NasKar2
npm install devtools-protocol
npm install puppeteer@latest
fixes your erors
BUT
npm start
gives this:
2022-02-05T21:47:38+00:00 INFO FlareSolverr v2.2.0
2022-02-05T21:47:38+00:00 INFO Testing web browser installation...
2022-02-05T21:47:38+00:00 ERROR Error: Unsupported platform: freebsd
at assert (/usr/local/share/FlareSolverr/node_modules/puppeteer/lib/cjs/puppeteer/common/assert.js:26:15)
at BrowserFetcher.setPlatform (/usr/local/share/FlareSolverr/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:180:36)
at new BrowserFetcher (/usr/local/share/FlareSolverr/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:164:14)
at FirefoxLauncher._updateRevision (/usr/local/share/FlareSolverr/node_modules/puppeteer/lib/cjs/puppeteer/node/Launcher.js:305:36)
at FirefoxLauncher.launch (/usr/local/share/FlareSolverr/node_modules/puppeteer/lib/cjs/puppeteer/node/Launcher.js:254:20)
at async create (/usr/local/share/FlareSolverr/dist/services/sessions.js:99:19)
at async testWebBrowserInstallation (/usr/local/share/FlareSolverr/dist/services/sessions.js:71:21)

@Zamana
Copy link
Author

Zamana commented Feb 6, 2022

I'm sorry, but I'm unable to make version 2.2.0 works in FreeBSD.

My instructions are valid only for version 2.1.0.

The article is updated about this.

@Zamana
Copy link
Author

Zamana commented Feb 7, 2022

Version 2.2.1 doesn't work either.

The only one until now is 2.1.0, and unfortunately I know nothing about Node...

@qudiqudi
Copy link

qudiqudi commented Feb 7, 2022

Can confirm following your instructions, FlareSolverr is working inside a jail.
Though this discussion over at puppeteer seems interesting: puppeteer/puppeteer#1049 (comment)
According to the changelogs, FlareSolverr got bumped to node16, there is a package for that in ports. I think it's really only the missing os compatibility check for FreeBSD in puppeteer that stops version 2.2.+ from working.

@Zamana
Copy link
Author

Zamana commented Feb 7, 2022

I tried with Node16, and the result was the same.

@qudiqudi
Copy link

qudiqudi commented Feb 7, 2022

Well yes, the OS check problem in puppeteer still persists. And puppeteer 1.2.0 isn't compatible with 2.2.+

@HendricksonPlex
Copy link

Is there a reason you are using Chromium vs Firefox, which FlareSolverr installs by default?

@Limych
Copy link

Limych commented Feb 11, 2022

Thank you very much!
It works! ❤️

My script for automated installation: https://gist.github.com/Limych/eeef7347baab216d73dfb83a6247b676#file-freenas12-media-jail-sh-L75

@Zamana
Copy link
Author

Zamana commented Feb 11, 2022

Is there a reason you are using Chromium vs Firefox, which FlareSolverr installs by default?

I confess that I can't remember quite well, but it seems that it didn't work with Firefox in FreeBSD.

Did you try with Firefox with success?

@Zamana
Copy link
Author

Zamana commented Feb 11, 2022

Thank you very much!
It works! ❤️

Thank you!
Very nice work with the automation.

@callaa3
Copy link

callaa3 commented Feb 14, 2022

@Limych thanks for that, need to move "git checkout tags/v2.1.0 -f" after "cd /usr/local/share/FlareSolverr"

@Limych
Copy link

Limych commented Feb 14, 2022

@callaa3 Thanks ❤️

@Limych
Copy link

Limych commented Feb 14, 2022

@Zamana
I'm not sure (I'm not test it), but I think it's possible to replace

npm install puppeteer@1.2.0
npm install puppeteer

with one command:

npm install puppeteer@1.20.0

@onlyrootathome
Copy link

onlyrootathome commented Feb 16, 2022

@Zamana do you know how to enable flaresolverr logs in TrueNAS jail install. I've found a guide for docker, but not sure how to do it after following this guide. (Thank you for this guide!)

Reason I ask - I'm getting errors and want to read the logs to diagnose the issue:

An error occurred while testing this indexer
Exception: FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Error: Unable to process browser request. Error: Maximum timeout reached. maxTimeout=55000 (ms): FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Error: Unable to process browser request. Error: Maximum timeout reached. maxTimeout=55000 (ms)

@Zamana
Copy link
Author

Zamana commented Feb 16, 2022

Hi!

FlareSolverr reads the environment variable LOG_LEVEL in order to set the log verbosity.

The valid values are:

['error', 'warn', 'info', 'verbose', 'debug']

You can set it through command line (if you are running through command line) or by changing the init script (check the flaresolverr_precmd() function).

@jamaroney
Copy link

jamaroney commented Mar 5, 2022

Hi,

I followed your instructions exactly, but when I finally run "npm start", I get this:

> flaresolverr@2.1.0 start /usr/local/share/FlareSolverr
> node ./dist/server.js

2022-01-01T20:32:20-03:00 INFO FlareSolverr v2.1.0
2022-01-01T20:32:20-03:00 INFO Testing web browser installation...

and that's it. I waited about 15 minutes, and it never got past "Testing web browser installation"

While "testing", my processes show three instances of chrome running, and never less than 80%.

I then setup the rc.d file and added "flaresolverr_enable=YES" to my rc.conf file. After rebooting my machine, there were again three instances of chrome running.

"service flaresolverr status" states that it's running, but "netstat -an | grep 8191" shows nothing.

Any thoughts?

@Zamana
Copy link
Author

Zamana commented Mar 5, 2022

I'm really sorry, but I'm afraid to say that I don't know what could possible going on.

I know nothing about Node/NPM or any other technology used by Flaresolverr project. The only thing I did was discover how to make Flaresolverr works inside a FreeNAS jail.

In my specific case, that's what htop shows me when Flaresolverr is running:

image

I guess you could try to post your issue in the Flaresolverr community, but maybe they will repeat the "mantra" that I'm hearing everywhere: "if you use a 'niche OS', you're on your own". Sad.

Regards.

@jamaroney
Copy link

jamaroney commented Mar 6, 2022

I should have been clearer. I was attempting to install FlareSolverr onto a headless XigmaNAS (formerly NAS4FREE) server running FREEBSD 12.3.0.4. I finally figured it out, but also want to note some other problems I had to solve as well, and a possible remaining problem.

After unzipping/renaming FlareSolverr 2.10 and moving it to /usr/local/share, I had to modify some of the later commands, as I was constantly getting error messages that I didn't have enough disk space (untrue). The modifications/additions are bold:


setenv PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
setenv PUPPETEER_EXECUTABLE_PATH /usr/local/bin/chrome
npm install --cache /tmp/empty-cache
npm audit fix (sometimes this was needed, sometimes not)
node node_modules/puppeteer/install.js
npm install puppeteer@1.2.0 --cache /tmp/empty-cache
npm audit fix
(There was no need to perform the command afterward "npm install puppeteer")
npm run build


It was at this point, however, that if I ran "npm start" I encountered the problem I described in my previous post. What solved it was to modify the last line of /usr/local/bin/chrome from
exec /usr/local/share/chromium/chrome ${1+"$@"}
to
exec /usr/local/share/chromium/chrome --headless --disable-gpu ${1+"$@"}

Afterwards everything seemed to run fine. However, in Jackett I attempted to test iDope and got a message saying "The cookies provided by FlareSolverr are not valid." I wanted to see if the problem was limited to iDope, but was unable to find another indexer on Jackett that uses Cloudflare. So at this point I don't know if there's an underlying problem in my FlareSolverr install or elsewhere.

@Zamana
Copy link
Author

Zamana commented Mar 6, 2022

Hi!

Congratulations for your further work on this subject. I guess that this will help several people.

Regarding the use of Flaresolverr itself, if you follow their issues page on Github, there are a lot of complains about it. It seems that Cloudfare changed something recently regarding cookies or something like this, and this is affecting the Flaresolverr's ability to bypass Cloudfare's protection.

One site that I know that uses Cloudfare is 1337x.

@thejacobsons
Copy link

Hello all! I was banging my head against the wall on this one! A few notes in case you are having a problem completing the install:

  1. No need to install git-tiny to git clone
  2. Since v2.1.0 is the version needed, use the following steps:
    fetch https://github.com/FlareSolverr/FlareSolverr/archive/refs/tags/v2.1.0.zip
    unzip v2.1.0.zip
    mv FlareSolverr-2.1.0 FlareSolverr
  3. Run the npm installs BEFORE node command

Anyhow, I hope this helps you going forward. Since I did not know if using puppeteer@latest would cause an issue, I used the version listed above. Now I have a successful npm start! :)

@Zamana
Copy link
Author

Zamana commented May 17, 2022

The rationale behind git is to allow you to be able to update the application directly from the repository, but of course that's optional.

Thanks for the additional remarks.

@ak1ra-komj
Copy link

Because FreeBSD lacks headless chromium builds, a large number of X-related packages will be introduced when installing chromium packages, I am curious why FreeBSD Ports does not build headless chromium?

@BobThompson
Copy link

Is there a way to fix chromium not installing? I have tried both quarterly and latest to no avail.

pkg install chromium
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
pkg: No packages available to install matching 'chromium' have been found in the repositories

@Zamana
Copy link
Author

Zamana commented Mar 20, 2023

Hi!

Thanks for commenting.

I confess that I did give up on running FlareSolverr on FreeBSD days after writing this article.

Now I have a small Alpine Linux VM inside TrueNAS where FlareSolverr is running through a docker. It's much better and avoids all the hassle...

I'm sorry, but I don't know how to solve this issue.

Regards.

@zajc3w
Copy link

zajc3w commented Mar 20, 2023

Hi!

Thanks for commenting.

I confess that I did give up on running FlareSolverr on FreeBSD days after writing this article.

Now I have a small Alpine Linux VM inside TrueNAS where FlareSolverr is running through a docker. It's much better and avoids all the hassle...

I'm sorry, but I don't know how to solve this issue.

Regards.

TBH with TruenasScale reaching stability there is no need.
just switch from freenas/truenas core to Scale and use truecharts for all your apps,.
Flaresolverr installs in under a minute, once you get hang of truecharts, and that took me an one evening.
well worth it/ it does require ssd's for apps(some mwill take ages to deploy without them) but still runs on ancient hardware(i3-4170 in my case)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment