实战 Web 应用容器化

这篇文章会介绍如何将后端、前端和网关通通使用 Docker 容器进行运行，并最终使用 DockerCompose 进行容器编排。

技术栈

前端

React
Ant Design

后端

Go
Iris

网关

Nginx
OpenResty
Lua
企业微信

后端构建 api

这里虽然我们写了 EXPOSE 4182，这个只用在测试的时候，生产环境实际上我们不会将后端接口端口进行暴露，而是通过容器间的网络进行互相访问，以及最终会使用 Nginx 进行转发。

FROM golang:1.15.5

LABEL maintainer="K8sCat <[email protected]>"

EXPOSE 4182

ENV GOPROXY=https://goproxy.cn,direct \
    GO111MODULE=on

WORKDIR /go/src/github.com/k8scat/containerized-app/api

COPY . .

RUN go mod download && \
go build -o api main.go && \
chmod +x api

ENTRYPOINT [ "./api" ]

前端构建 web

这里值得一提的是，因为前端肯定会去调用后端接口，而且这个接口地址是根据部署而改变，所以这里我们使用了 ARG 指令进行设置后端的接口地址，这样我们只需要在构建镜像的时候传入 --build-arg REACT_APP_BASE_URL=https://example.com/api 就可以调整后端接口地址了，而不是去改动代码。

还有一点，有朋友肯定会发现这里同时使用到了 Entrypoint 和 CMD，这是为了可以在运行的时候调整前端的端口，但实际上我们这里没必要去调整，因为这里最终也是用 Nginx 进行转发。

FROM node:lts

LABEL maintainer="[email protected]"

WORKDIR /web

COPY . .

ARG REACT_APP_BASE_URL

RUN npm config set registry https://registry.npm.taobao.org && \
npm install && \
npm run build && \
npm install -g serve

ENTRYPOINT [ "serve", "-s", "build" ]
CMD [ "-l", "3214" ]

网关构建 gateway

Nginx 配置

这里我们就分别设置了后端和前端的上游，然后设置 location 规则进行转发。

这里有几个点可以说一下：

通过 set_by_lua 获取容器的环境变量，最终在运行的时候通过设置 environment 设置这些环境变量，更加灵活
server_name 使用到了 $hostname，运行时需要设置容器的 hostname
ssl_certificate 和 ssl_certificate_key 不能使用变量设置
加载 gateway.lua 脚本实现企业微信的网关认证

upstream web {
    server ca-web:3214;
}

upstream api {
	server ca-api:4182;
}

server {
	set_by_lua $corp_id 'return os.getenv("CORP_ID")';
	set_by_lua $agent_id 'return os.getenv("AGENT_ID")';
	set_by_lua $secret 'return os.getenv("SECRET")';
	set_by_lua $callback_host 'return os.getenv("CALLBACK_HOST")';
	set_by_lua $callback_schema 'return os.getenv("CALLBACK_SCHEMA")';
	set_by_lua $callback_uri 'return os.getenv("CALLBACK_URI")';
	set_by_lua $logout_uri 'return os.getenv("LOGOUT_URI")';
	set_by_lua $token_expires 'return os.getenv("TOKEN_EXPIRES")';
	set_by_lua $use_secure_cookie 'return os.getenv("USE_SECURE_COOKIE")';

	listen 443 ssl http2;
	server_name $hostname;
	resolver 8.8.8.8;
	ssl_certificate /certs/cert.crt;
	ssl_certificate_key /certs/cert.key;
	ssl_session_cache shared:SSL:1m;
	ssl_session_timeout 5m;
	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
	ssl_ciphers AESGCM:HIGH:!aNULL:!MD5;
	ssl_prefer_server_ciphers on;
	lua_ssl_verify_depth 2;
    lua_ssl_trusted_certificate /etc/pki/tls/certs/ca-bundle.crt;

	if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
		set $year $1;
		set $month $2;
		set $day $3;
	}
	access_log logs/access_$year$month$day.log main;
	error_log logs/error.log;

	access_by_lua_file "/usr/local/openresty/nginx/conf/gateway.lua";

	location ^~ /gateway {
        root   html;
        index  index.html index.htm;
    }

	location ^~ /api {
        proxy_pass http://api;
        proxy_read_timeout 3600;
        proxy_http_version 1.1;
        proxy_set_header X_FORWARDED_PROTO https;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_set_header Connection "";
    }

	location ^~ / {
        proxy_pass http://web;
        proxy_read_timeout 3600;
        proxy_http_version 1.1;
        proxy_set_header X_FORWARDED_PROTO https;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_set_header Connection "";
    }

	error_page 500 502 503 504 /50x.html;
	location = /50x.html {
		root html;
	}
}

server {
	listen 80;
	server_name $hostname;

	location / {
		rewrite ^/(.*) https://$server_name/$1 redirect;
	}
}

Dockerfile

FROM openresty/openresty:1.19.3.1-centos

LABEL maintainer="K8sCat <[email protected]>"

COPY gateway.conf /etc/nginx/conf.d/gateway.conf
COPY gateway.lua /usr/local/openresty/nginx/conf/gateway.lua
COPY nginx.conf /usr/local/openresty/nginx/conf/nginx.conf

# Install lua-resty-http
RUN /usr/local/openresty/luajit/bin/luarocks install lua-resty-http

Lua 实现基于企业微信的网关认证

这里面的一些配置参数都是通过获取 Nginx 设置的变量。

local json = require("cjson")
local http = require("resty.http")

local uri = ngx.var.uri
local uri_args = ngx.req.get_uri_args()
local scheme = ngx.var.scheme

local corp_id = ngx.var.corp_id
local agent_id = ngx.var.agent_id
local secret = ngx.var.secret
local callback_scheme = ngx.var.callback_scheme or scheme
local callback_host = ngx.var.callback_host
local callback_uri = ngx.var.callback_uri
local use_secure_cookie = ngx.var.use_secure_cookie == "true" or false
local callback_url = callback_scheme .. "://" .. callback_host .. callback_uri
local redirect_url = callback_scheme .. "://" .. callback_host .. ngx.var.request_uri
local logout_uri = ngx.var.logout_uri or "/logout"
local token_expires = ngx.var.token_expires or "7200"
token_expires = tonumber(token_expires)

local function request_access_token(code)
    local request = http.new()
    request:set_timeout(7000)
    local res, err = request:request_uri("https://qyapi.weixin.qq.com/cgi-bin/gettoken", {
        method = "GET",
        query = {
            corpid = corp_id,
            corpsecret = secret,
        },
        ssl_verify = true,
    })
    if not res then
        return nil, (err or "access token request failed: " .. (err or "unknown reason"))
    end
    if res.status ~= 200 then
        return nil, "received " .. res.status .. " from https://qyapi.weixin.qq.com/cgi-bin/gettoken: " .. res.body
    end
    local data = json.decode(res.body)
    if data["errcode"] ~= 0 then
        return nil, data["errmsg"]
    else
        return data["access_token"]
    end
end

local function request_user(access_token, code)
    local request = http.new()
    request:set_timeout(7000)
    local res, err = request:request_uri("https://qyapi.weixin.qq.com/cgi-bin/user/getuserinfo", {
        method = "GET",
        query = {
            access_token = access_token,
            code = code,
        },
        ssl_verify = true,
    })
    if not res then
        return nil, "get profile request failed: " .. (err or "unknown reason")
    end
    if res.status ~= 200 then
        return nil, "received " .. res.status .. " from https://qyapi.weixin.qq.com/cgi-bin/user/getuserinfo"
    end
    local userinfo = json.decode(res.body)
    if userinfo["errcode"] == 0 then
        if userinfo["UserId"] then
            res, err = request:request_uri("https://qyapi.weixin.qq.com/cgi-bin/user/get", {
                method = "GET",
                query = {
                    access_token = access_token,
                    userid = userinfo["UserId"],
                },
                ssl_verify = true,
            })
            if not res then
                return nil, "get user request failed: " .. (err or "unknown reason")
            end
            if res.status ~= 200 then
                return nil, "received " .. res.status .. " from https://qyapi.weixin.qq.com/cgi-bin/user/get"
            end
            local user = json.decode(res.body)
            if user["errcode"] == 0 then
                return user
            else
                return nil, user["errmsg"]
            end
        else
            return nil, "UserId not exists"
        end
    else
        return nil, userinfo["errmsg"]
    end
end

local function is_authorized()
    local headers = ngx.req.get_headers()
    local expires = tonumber(ngx.var.cookie_OauthExpires) or 0
    local user_id = ngx.unescape_uri(ngx.var.cookie_OauthUserID or "")
    local token = ngx.var.cookie_OauthAccessToken or ""
    if expires == 0 and headers["OauthExpires"] then
        expires = tonumber(headers["OauthExpires"])
    end
    if user_id:len() == 0 and headers["OauthUserID"] then
        user_id = headers["OauthUserID"]
    end
    if token:len() == 0 and headers["OauthAccessToken"] then
        token = headers["OauthAccessToken"]
    end
    local expect_token = callback_host .. user_id .. expires
    if token == expect_token and expires then
        if expires > ngx.time() then
            return true
        else
            return false
        end
    else
        return false
    end
end

local function redirect_to_auth()
    return ngx.redirect("https://open.work.weixin.qq.com/wwopen/sso/qrConnect?" .. ngx.encode_args({
        appid = corp_id,
        agentid = agent_id,
        redirect_uri = callback_url,
        state = redirect_url
    }))
end

local function authorize()
    if uri ~= callback_uri then
        return redirect_to_auth()
    end
    local code = uri_args["code"]
    if not code then
        ngx.log(ngx.ERR, "not received code from https://open.work.weixin.qq.com/wwopen/sso/qrConnect")
        return ngx.exit(ngx.HTTP_FORBIDDEN)
    end

    local access_token, request_access_token_err = request_access_token(code)
    if not access_token then
        ngx.log(ngx.ERR, "got error during access token request: " .. request_access_token_err)
        return ngx.exit(ngx.HTTP_FORBIDDEN)
    end

    local user, request_user_err = request_user(access_token, code)
    if not user then
        ngx.log(ngx.ERR, "got error during profile request: " .. request_user_err)
        return ngx.exit(ngx.HTTP_FORBIDDEN)
    end
    ngx.log(ngx.ERR, "user id: " .. user["userid"])

    local expires = ngx.time() + token_expires
    local cookie_tail = "; version=1; path=/; Max-Age=" .. expires
    if use_secure_cookie then
        cookie_tail = cookie_tail .. "; secure"
    end

    local user_id = user["userid"]
    local user_token = callback_host .. user_id .. expires

    ngx.header["Set-Cookie"] = {
        "OauthUserID=" .. ngx.escape_uri(user_id) .. cookie_tail,
        "OauthAccessToken=" .. ngx.escape_uri(user_token) .. cookie_tail,
        "OauthExpires=" .. expires .. cookie_tail,
    }
    return ngx.redirect(uri_args["state"])
end

local function handle_logout()
    if uri == logout_uri then
        ngx.header["Set-Cookie"] = "OauthAccessToken==deleted; path=/; expires=Thu, 01 Jan 1970 00:00:00 GMT"
        --return ngx.redirect("/")
    end
end

handle_logout()
if (not is_authorized()) then
    authorize()
end

使用 DockerCompose 进行容器编排

这里需要讲几个点：

设置前端的 args 可以在前端构建时传入后端接口地址
设置网关的 hostname 可以设置网关容器的 hostname
设置网关的 environment 可以传入相关配置
最终运行时只有网关层进行暴露端口

version: "3.8"

services:
  api:
    build: ./api
    image: ca-api:latest
    container_name: ca-api

  web:
    build:
      context: ./web
      args:
        REACT_APP_BASE_URL: https://example.com/api
    image: ca-web:latest
    container_name: ca-web
    
  gateway:
    build: ./gateway
    image: ca-gateway:latest
    hostname: example.com
    volumes:
      - ./gateway/certs/fullchain.pem:/certs/cert.crt
      - ./gateway/certs/privkey.pem:/certs/cert.key
    ports:
      - 80:80
      - 443:443
    environment:
      - CORP_ID=
      - AGENT_ID=
      - SECRET=
      - CALLBACK_HOST=example.com
      - CALLBACK_SCHEMA=https
      - CALLBACK_URI=/gateway/oauth_wechat
      - LOGOUT_URI=/gateway/oauth_logout
      - TOKEN_EXPIRES=7200
      - USE_SECURE_COOKIE=true
    container_name: ca-gateway

技术栈#

前端#

后端#

网关#

后端构建 api#

前端构建 web#

网关构建 gateway#

Nginx 配置#

Dockerfile#

Lua 实现基于企业微信的网关认证#

使用 DockerCompose 进行容器编排#

开源代码#

技术栈

前端

后端

网关