How to Set Proxies in Go Guide

Proxies are crucial for various network operations, such as accessing restricted content, enhancing privacy, and debugging network traffic. The global proxy market has grown significantly in the last few years, making it an essential part of modern software ecosystems.

Overview of Proxy Configuration in Go

Go supports proxy configuration through its standard library and additional packages. For example, Go’s standard library includes the net/http package, which offers built-in support for proxy configuration. Additionally, third-party packages like SOCKS, goproxy, oxy and NTML can be used to define more advanced proxy configurations.

Here’s a basic example of setting up a proxy using the standard library:

package main

import (
	"net/http"
	"net/url"
)

func main() {
	proxyURL, _ := url.Parse("http://your-proxy-url:port")
	transport := &http.Transport{
		Proxy: http.ProxyURL(proxyURL),
	}
	client := &http.Client{
		Transport: transport,
	}

	resp, err := client.Get("http://example.com")
	if err != nil {
		// handle error
	}
	defer resp.Body.Close()
	// process response
}

Setting Proxies for HTTP Clients

HTTP proxies are designed to handle HTTP traffic, making them ideal for web-based applications. They can interpret and manage HTTP requests and responses, allowing for features like URL filtering, content caching, and load balancing. Additionally, they handle specific HTTP headers and methods, providing fine-grained control over web requests.

In this section, we’ll show you how to set up HTTP proxies with Go’s net/http package and then compare it with HTTPS and SOCKS proxy configurations later.

Step 1: Creating a Proxy Function

First, you need to define a function that returns the proxy URL. The HTTP client’s transport layer will use this function to route requests through the proxy.

func ProxyFunc(req *http.Request) (*url.URL, error) {
    return url.Parse("http://your-proxy-server.com:8080")
}

Step 2: Configuring the Transport Layer of the HTTP Client

Next, create a custom http transport object and set the Proxy field to the proxy function:

transport := &http.Transport{
    Proxy: ProxyFunc,
}

client := &http.Client{
    Transport: transport,
}

Now, any requests made using this client will be routed through the specified proxy server.

Here’s a complete example:

package main

import (
    "net/http"
    "net/url"
)

func ProxyFunc(req *http.Request) (*url.URL, error) {
    return url.Parse("http://your-proxy-server.com:8080") // Parsing the proxy URL
}

func main() {
    transport := &http.Transport{ // Creating a new Transport object
        Proxy: ProxyFunc,  //  Setting the transport objects Proxy field to the parsed proxy URL
    }

    client := &http.Client{ // Create a new http.Client instance with the configured transport
        Transport: transport,
    }

    resp, err := client.Get("http://example.com")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
}

Step 3: Defining a Custom Proxy Function

Sometimes, you may need more control over which requests are proxied. In such situations, you can create a custom proxy function that sets the proxy conditionally based on request attributes.

func CustomProxyFunc(req *http.Request) (*url.URL, error) {
    if req.URL.Host == "example.com" {
        return url.Parse("http://your-proxy-server.com:8080")
    }
    return nil, nil
}

Here’s how to use this custom proxy function:

package main

import (
    "net/http"
    "net/url"
)

func CustomProxyFunc(req *http.Request) (*url.URL, error) {
    if req.URL.Host == "example.com" {
        return url.Parse("http://your-proxy-server.com:8080")
    }
    return nil, nil
}

func main() {
    transport := &http.Transport{
        Proxy: CustomProxyFunc,
    }

    client := &http.Client{
        Transport: transport,
    }

    resp, err := client.Get("http://example.com")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
}

This custom proxy function checks if the request should go through a proxy based on the host. If so, it returns the proxy URL; otherwise, it returns nil to indicate a direct connection.

Setting Proxies for HTTPS Connections

HTTPS proxies, also known as tunnelling proxies, handle encrypted data and ensure the security and integrity of the connection. You can implement HTTPS proxies using Go’s net/http package.

Note: Tunnelling is a technique HTTPS proxies use to establish a secure connection between the client and the target server. HTTPS proxies use the CONNECT method to establish a tunnel, allowing encrypted data to pass through without being decrypted by the proxy.

Step 1: Import Necessary Packages

import (
    "context"
    "crypto/tls"
    "fmt"
    "io"
    "net"
    "net/http"
    "net/url"
    "time"
)

• context: Manage the DialContext function, which uses a context for timeouts and cancellations.
• crypto/tls:Configure TLS settings for secure connections.
• fmt: String formatting and printing (e.g., creating the CONNECT request string).
• io: Reading the response body from the HTTP request.
• net: Provides functions for network connections, such as dialling to the proxy server.
• net/http: Creating the HTTP client and making requests.
• net/url: Used to parse and manage URLs.
• time: Setting timeouts on network operations.

Step 2: Parse the proxy URL

proxyURL, err := url.Parse("http://your-proxy-url:port")
if err != nil {
    // handle error
}

Step 3: Define the DialContext Function

Configure the Transport layer to use a custom DialContext function. This function handles the creation of the tunnel for secure connections.

transport := &http.Transport{
        Proxy: http.ProxyURL(proxyURL),
        DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
            // Connect to the proxy server
            conn, err := (&net.Dialer{
                Timeout:   30 time.Second,
                KeepAlive: 30  time.Second,
            }).DialContext(ctx, network, proxyURL.Host)
            if err != nil {
                return nil, err
            }

            // Send CONNECT request to the proxy
            connectReq := fmt.Sprintf("CONNECT %s HTTP/1.1\r\nHost: %s\r\n\r\n", addr, addr)
            _, err = conn.Write([]byte(connectReq))
            if err != nil {
                conn.Close()
                return nil, err
            }

            // Read the proxy's response (usually a 200 OK)
            buf := make([]byte, 4096)
            n, err := conn.Read(buf)
            if err != nil {
                conn.Close()
                return nil, err
            }
            if n == 0 || !httpReadOK(buf[:n]) {
                conn.Close()
                return nil, fmt.Errorf("failed to CONNECT to proxy, response: %s", buf[:n])
            }

            return conn, nil
        },
        TLSHandshakeTimeout: 10 * time.Second,
    }

Step 4: Create the HTTP Client

    client := &http.Client{
        Transport: transport,
    }

Step 5: Make an HTTPS Request

Use the configured client to make an HTTPS request and handle the response:

    resp, err := client.Get("https://example.com")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    body, err := io.ReadAll(resp.Body)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(body))
}

Here is the complete code with all the steps combined:

package main

import (
    "context"
    "crypto/tls"
    "fmt"
    "io"
    "net"
    "net/http"
    "net/url"
    "time"
)

func main() {
    proxyURL, _ := url.Parse("http://your-https-proxy.com:8080")
    transport := &http.Transport{
        Proxy: http.ProxyURL(proxyURL),
        DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
            // Connect to the proxy server
            conn, err := (&net.Dialer{
                Timeout:   30 * time.Second,
                KeepAlive: 30 * time.Second,
            }).DialContext(ctx, network, proxyURL.Host)
            if err != nil {
                return nil, err
            }

            // Send CONNECT request to the proxy
            connectReq := fmt.Sprintf("CONNECT %s HTTP/1.1\r\nHost: %s\r\n\r\n", addr, addr)
            _, err = conn.Write([]byte(connectReq))
            if err != nil {
                conn.Close()
                return nil, err
            }

            // Read the proxy's response (usually a 200 OK)
            buf := make([]byte, 4096)
            n, err := conn.Read(buf)
            if err != nil {
                conn.Close()
                return nil, err
            }
            if n == 0 || !httpReadOK(buf[:n]) {
                conn.Close()
                return nil, fmt.Errorf("failed to CONNECT to proxy, response: %s", buf[:n])
            }

            return conn, nil
        },
        TLSHandshakeTimeout: 10 * time.Second,
        TLSClientConfig:     &tls.Config{InsecureSkipVerify: true}, // For testing purposes only; do not use in production
    }

    client := &http.Client{
        Transport: transport,
    }

    resp, err := client.Get("https://example.com")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    body, err := io.ReadAll(resp.Body)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(body))
}

// Helper Function to Check Proxy Response

func httpReadOK(data []byte) bool {
    return len(data) >= 12 && string(data[:12]) == "HTTP/1.1 200"
}

You can further improve this code by using environment variables for proxy settings. For example, HTTP_PROXY and HTTPS_PROXY environment variables can be used to specify the proxy URLs for HTTP and HTTPS connections.

export HTTP_PROXY=http://your-http-proxy-url.com:8080
export HTTPS_PROXY=http://your-https-proxy-url.com:8080

Setting SOCKS Proxies

SOCKS proxies operate at a lower level (Network Layer) than HTTP proxies. So they can handle any network traffic without being limited to HTTP. This flexibility makes SOCKS proxies useful for applications requiring multiple protocol support. SOCKS proxies are often used to bypass firewalls or access restricted content and provide anonymity by masking the user’s IP address.

To set a SOCKS proxy in Go, you can use the golang.org/x/net/proxy package, which provides a convenient way to create and configure various proxies.

Step 1: Import the Necessary Packages

package main

import (
    "golang.org/x/net/proxy"
    "net/http"
)

Step 2: Create a SOCKS5 Proxy Dialer

Use the proxy.SOCKS5 function to create a SOCKS5 proxy dialer. This function takes the network type, the address of the SOCKS proxy server, authentication, and a forwarding proxy.

func main() {
    dialer, err := proxy.SOCKS5("tcp", "your-socks-proxy.com:1080", nil, proxy.Direct)
    if err != nil {
        panic(err)
    }

Step 3: Configure the HTTP Transport:

Set the Dial field of the http.Transport struct to use the SOCKS5 dialer. This ensures that all connections made by the http.Client will go through the specified SOCKS proxy server.

   transport := &http.Transport{
        Dial: dialer.Dial,
    }

Step 4: Create an HTTP Client with the Configured Transport

Initialize the http.Client to use the custom transport configuration that routes through the SOCKS proxy.

   client := &http.Client{
        Transport: transport,
    }

Step 5: Make a Request using the HTTP Client

Use the configured client to make a request. This ensures that the request goes through the SOCKS proxy.

    resp, err := client.Get("http://example.com")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

Here’s the complete code with all the steps combined:

package main

import (
    "golang.org/x/net/proxy"
    "net/http"
)

func main() {
    dialer, err := proxy.SOCKS5("tcp", "your-socks-proxy.com:1080", nil, proxy.Direct)
    if err != nil {
        panic(err)
    }

    transport := &http.Transport{
        Dial: dialer.Dial,
    }

    client := &http.Client{
        Transport: transport,
    }

    resp, err := client.Get("http://example.com")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
}

Handling Authentication

Some proxies require authentication to ensure only authorized users can access network resources. Proper authentication handling is also crucial for network security.

Basic Authentication

One way to handle basic authentication is by including the credentials directly in the proxy URL. This method is straightforward but has security implications, as the credentials are stored in plaintext within the code.

package main

import (
    "net/http"
    "net/url"
)

func ProxyFunc(req *http.Request) (*url.URL, error) {
    return url.Parse("http://username:[email protected]:8080")
}

func main() {
    transport := &http.Transport{
        Proxy: ProxyFunc,
    }

    client := &http.Client{
        Transport: transport,
    }

    resp, err := client.Get("http://example.com")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
}

Advanced Authentication Methods

Certain proxy servers require more complex authentication mechanisms, such as NTLM (NT LAN Manager) or Kerberos. These methods often involve additional libraries or packages to handle the authentication process correctly.

To handle NTLM authentication for a proxy server, you can use the github.com/Azure/go-ntlmssp package.

package main

import (
    "github.com/Azure/go-ntlmssp"
    "net/http"
    "net/url"
)

func main() {
    proxyURL, _ := url.Parse("http://your-proxy-server.com:8080")
    transport := &http.Transport{
        Proxy: http.ProxyURL(proxyURL),
        DialContext: (&net.Dialer{
            Timeout:   30 * time.Second,
            KeepAlive: 30 * time.Second,
        }).DialContext,
        TLSHandshakeTimeout: 10 * time.Second,
    }

    client := &http.Client{
        Transport: ntlmssp.Negotiator{
            RoundTripper: transport,
        },
    }

    resp, err := client.Get("http://example.com")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
}

Conclusion

This article is a complete guide to configuring and using proxies in Go. It explained how to configure HTTP, HTTPS and SOCKS proxies with Go, including best practices like using environment variables and authentication. Setting up proxies in Go can significantly enhance your application’s network capabilities, providing benefits such as accessing restricted content, improving privacy, and facilitating network traffic debugging. Go’s libraries and packages make the process straightforward and effective, enabling you to build robust and secure network applications.

Looking for a proxy provider? Read our review of the best proxy providers.

FAQs