How to set up your own proxy server

What you get, and what you are actually building

A proxy server of your own is one machine with one public IP that forwards HTTP requests on your behalf. That gives you three things worth having: a fixed address you control, so an API or an office firewall can whitelist it; a jump host, so traffic from several of your own machines leaves the internet from one predictable place; and optionally a cache, which saves real bandwidth if many clients pull the same large files.

What it does not give you is a different identity every request. One VPS is one datacentre address, and that matters later — but first, the build.

You need a small VPS (the cheapest tier on any provider is plenty), a current Debian or Ubuntu image, root or sudo, and the public IP address of the machine you will connect from.

Squid on a Linux VPS: install

Squid is the standard forward proxy on Linux. It is in every distribution's repository, so there is nothing to compile:

sudo apt update
sudo apt install -y squid apache2-utils
sudo cp /etc/squid/squid.conf /etc/squid/squid.conf.orig

apache2-utils is there only for htpasswd, which creates the password file. The stock squid.conf that ships with the package is around 8,000 lines of comments; keep the copy for reference and write your own from scratch, because a configuration you can read in one screen is a configuration you can audit.

Create a user before touching the config, and make the file readable by the proxy account only:

sudo htpasswd -c /etc/squid/passwd alice
sudo chown proxy:proxy /etc/squid/passwd
sudo chmod 640 /etc/squid/passwd

Drop the -c flag for every user after the first, or you will overwrite the file.

A minimal squid.conf with basic authentication

Replace /etc/squid/squid.conf with this. Every line has a job; nothing here is decoration:

http_port 3128

auth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid/passwd
auth_param basic realm PyProxy private proxy
auth_param basic children 5
auth_param basic credentialsttl 2 hours

acl authenticated proxy_auth REQUIRED
acl SSL_ports port 443
acl Safe_ports port 80 443
acl CONNECT method CONNECT

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow authenticated
http_access deny all

# do not announce the proxy or the client to the target
via off
forwarded_for delete
request_header_access X-Forwarded-For deny all

cache deny all
coredump_dir /var/spool/squid
dns_v4_first on

The order of the http_access lines is the whole security model: Squid evaluates them top to bottom and stops at the first match, so deny all at the bottom means anything not explicitly allowed above is refused. Delete http_access allow authenticated and you have locked yourself out; delete http_access deny all and you have built an open proxy for the entire internet.

The basic_ncsa_auth path is distribution-specific. On Debian and Ubuntu it is /usr/lib/squid/basic_ncsa_auth; on RHEL-family systems it is /usr/lib64/squid/basic_ncsa_auth. Check with ls /usr/lib/squid/ before you restart, because a wrong path fails as "authentication is broken" rather than "file not found".

Basic auth sends the password base64-encoded, which is not encryption. It is fine for CONNECT tunnels to HTTPS sites, since the payload inside the tunnel is still TLS, but it does mean the proxy password itself is on the wire. Use a long random password and restrict the port.

Validate the file before restarting — Squid will tell you the line number of a mistake:

sudo squid -k parse
sudo systemctl restart squid
sudo systemctl status squid --no-pager

If you do want caching, add a cache_dir instead of cache deny all, then initialise the store with squid -z while the service is stopped:

cache_dir ufs /var/spool/squid 2048 16 256
maximum_object_size 128 MB

Opening the port, then testing with curl

Squid is now listening on 3128, but two firewalls usually stand in front of it: the one on the host and the one in your provider's control panel. Open both, and open them narrowly:

sudo ufw allow from 203.0.113.45 to any port 3128 proto tcp
sudo ufw allow OpenSSH
sudo ufw enable
sudo ufw status verbose

Replace 203.0.113.45 with the address you actually connect from. If your own IP changes daily and you must expose 3128 to the world, accept that scanners will find it within hours — authentication is then the only thing between them and your bandwidth bill, and your host will suspend the server if the proxy is abused.

Confirm the process is bound where you expect:

sudo ss -tlnp | grep 3128

Now, from your own machine rather than from the server, run the two requests that matter — a plain HTTP one and a CONNECT tunnel to HTTPS:

curl -x http://alice:PASSWORD@203.0.113.10:3128 http://httpbin.org/ip
curl -x http://alice:PASSWORD@203.0.113.10:3128 https://httpbin.org/ip

Both should return your server's IP address. If HTTP works and HTTPS does not, your Safe_ports or CONNECT rules are wrong. A 407 Proxy Authentication Required means Squid is running and reachable and only the credentials are wrong — which is good news, since it proves the network path. A hang with no response is almost always the firewall.

When something is unclear, the log is explicit about which rule fired:

sudo tail -f /var/log/squid/access.log
curl -sv -x http://alice:PASSWORD@203.0.113.10:3128 https://example.com -o /dev/null

3proxy, the lighter alternative

If you do not want a caching engine, 3proxy is a single small binary driven by one config file. It also speaks SOCKS5 natively, which Squid does not, and it happily runs several listeners at once. Build or install it, then write /etc/3proxy/3proxy.cfg:

nserver 1.1.1.1
nserver 8.8.8.8
nscache 65536
timeouts 1 5 30 60 180 1800 15 60
log /var/log/3proxy/3proxy.log D
rotate 7

users alice:CL:CHANGE_THIS_PASSWORD
auth strong
allow alice

proxy -p3128 -a
socks -p1080
flush

auth strong means username and password are required; allow alice is the access rule that follows it, and as with Squid the order matters. -a puts the HTTP proxy in anonymous mode so it does not add X-Forwarded-For. CL: stores the password in clear text in the file — use MD5: or NT: hashes in anything but a test, and keep the file at mode 600.

Then run it under systemd and test exactly as above. 3proxy costs a few megabytes of RAM and starts instantly, which makes it the better fit on a 512 MB box or inside a container.

Where a server of your own stops being the answer

This is the part most tutorials leave out, and it is the reason people set up Squid twice. Your proxy works perfectly and then a target site starts serving captchas, 403s, or subtly different pages. Nothing is broken. The site simply looked up your IP, saw that it belongs to a hosting provider rather than to a household broadband subscriber, and treated it accordingly.

So split the work by what it needs:

• A fixed, controlled address — whitelisting, a jump host, an internal cache, hiding your office range from a partner API: self-host. Squid is the right tool and this guide is the whole job.
• Many addresses, or residential addresses — price monitoring, ad verification, market research, anything on a well-defended site: you need a pool, and building one is not a Squid problem. That is what a gateway is for.

A residential gateway looks the same to your code — one hostname, one port, basic auth, so any client that already talks to your Squid box talks to it unchanged:

curl -x http://USERNAME-country-us:PASSWORD@gw.pyproxy.com:1111 https://httpbin.org/ip

The options travel in the username: -country-us picks the exit country, and adding a session token holds one address for about thirty minutes when a site needs your requests to look like one continuous visit. PyProxy residential traffic is HTTP and HTTPS, billed per gigabyte rather than per address, so there is nothing to keep running and nothing to patch. Keep your own Squid for the jobs it is good at, and point the scraping at the gateway.

Questions people ask