Nginx的Web缓存服务与新浪网的开源NCACHE模块（1）

nginx的web缓存服务与新浪网的开源ncache模块

什么是web缓存

Web缓存位于内容源web服务器和客户端之间，当用户访问一个 URL时，web缓存服务器回去后端web源服务器取回要输出的内容，然后，当下一个请求到来时，如果访问的是相同的URL，web缓存服务器直接输出内容给客户端，而不是像源服务器再次发送请求。web缓存降低了内容源web服务器、数据库的负载，减轻了网络延迟，提高了用户的响应速度，增强了用户体验。
最著名的还要数Squid Cache，其主要在Unix一类系统运行。

Nginx的Web缓存服务

Nginx从0.7.48后支持类似于Squid的缓存模块。这个缓存是把URL及相关组合当做key，用md5算法对key进行希哈，得到硬盘上对应的希哈路径，从而将缓存内容保存在该目录内。支持任意URL链接。同时也支持404/301/302这样的非200状态码。
Nginx的Web缓存服务主要用于proxy_cache相关指令集和fastcgi相关指令集构成，前者用于反向代理时，对后端内容源进行缓存，后者主要用于对FastCDI的动态程序进行缓存。两者功能基本一样。

proxy_cache相关指令集

PathFinder

AI驱动的销售漏斗分析工具

下载

1、proxy_cache指令
语法：proxy_cache zone_name;
默认值：none
使用环境：http,server,location
该指令用于设置那个缓存区将被应用，zone_name的值为proxy_cache_path指令创建的缓存区明称。
2、proxy_cache_path指令
语法：proxy_cache_path path[levels=number] keys_z [max_size=size];
默认值：none
使用环境：HTTP
eg：
proxy_cache_path /data0/proxy_cache_dir levels=1:2 keys_z 500m inactive=1d max_size=30g;
注意该指令只能在http标签内配置，levels指定该缓存有两层hash目录，第一层为1个字母，第二层为2个字母，保存文件名类似于/data0/proxy_cache_dir/c/29/fdg35415fg35f4gsdf2g1535gh465h;key_zone参数用来为缓存区起名，500m指定内存空间大小为500MB；inactive的1d是如果缓存数据在1天之内没有被访问，将被删除；max_size的30g是指硬盘的缓存空间为30GB。
3proxy_cache_methods指令
语法：proxy_cache_methods [GET HEAD POST];
默认值：proxy_cache_methods GET HEAD;
使用环境：http，server，location
该指令用于设置用于缓存那些HTTP方法，默认缓存 HTTP GET/HEAD 方法，不缓存HTTP POST方法。
4proxy_cache_min_uses指令
语法：proxy_cache_min_uses the_number；
默认值：proxy_cache_min_uses 1；
使用环境：http，server，location
该指令设置缓存最小的使用次数，默认值是1.
5、proxy_cache_valid指令
语法：proxy_cache_valid reply_code [reply_code…]time;
默认值：none
使用环境：http，server，location
该指令用于对不同的返回状态码的URL设置不同的缓存时间，例如：
proxy_cache_valid 200 302 10m；
proxy_cache_valid 404 1m;
如果不指定状态吗，直接指定时间，则只有200、301、302状态的URL缓存5分钟。
6、proxy_cache_key指令
语法：proxy_cache_key line;
默认值：none
使用环境：http，server，location
该指令用来设置web缓存的key值，Nginx根据key值md5希哈存储缓存。一般根据‘$host(域名)、$request_uri(请求路径)’等组合变量合成proxy_cache_key.例如：proxy_cache_key "$host:$server_port$uri$is_args$args";

proxy_cache完整示例

su
yum -y install pcre//安装pcre
wget http://labs.frickle.com/files/ngx_cache_purge-2.3.tar.gz
tar zxvf ngx_cache_purge-2.3.tar.gz//获取nginx_cache_purge
cd nginx-1.6.3//进入你的nginx文件目录（nginx安装请参考前面的博客）
 ./configure --user=www --group=www --addmodule=../ngx_cache_purge-2.3 --prefix=/usr/local/webserver/nginx --with-http_stub_status_module --with-http_ssl_module

配置nginx.conf
cd /usr/local/webserver/nginx/conf

<span>#user  www www;</span><span>worker_processes</span><span>1</span>;

<span>#error_log  logs/error.log;</span><span>#error_log  logs/error.log  notice;</span><span>#error_log  logs/error.log  info;</span><span>#pid        logs/nginx.pid;</span><span>events</span> {
    <span>use</span><span>epoll</span>;
    <span>worker_connections</span><span>1024</span>;
}


<span>http</span> {
    <span>include</span>       mime.types;
    <span>default_type</span>  application/octet-stream;

    <span>#log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '</span><span>#                  '$status $body_bytes_sent "$http_referer" '</span><span>#                  '"$http_user_agent" "$http_x_forwarded_for"';</span><span>#access_log  logs/access.log  main;</span><span>#charset utf-8;</span><span>server_name_hash_bucket_size</span><span>128</span>;
    <span>client_header_buffer_size</span><span>32k</span>;
    <span>large_client_header_buffers</span><span>4</span><span>32k</span>;

    <span>sendfile</span><span>on</span>;
    <span>#tcp_nopush     on;</span><span>keepalive_timeout</span><span>30</span>;

    <span>tcp_nodely</span><span>on</span>;

    <span>proxy_temp_path</span> /data0/proxy_temp_path;

    <span>proxy_temp_path</span> /data0/proxy_temp_path levels=<span>1</span>:<span>2</span> key_z<span>200m</span> inactive=<span>1d</span> max_size=<span>30g</span>;
    <span>upstream</span> my_sever_pool{
        <span>server</span><span>192.168.1.2:80</span> weight=<span>1</span> max_fails=<span>2</span> fail_timeout=<span>30s</span>;
        <span>server</span><span>192.168.1.3:80</span> weight=<span>1</span> max_fails=<span>2</span> fail_timeout=<span>30s</span>;
        <span>server</span><span>192.168.1.4:80</span> weight=<span>1</span> max_fails=<span>2</span> fail_timeout=<span>30s</span>;

    }



    <span>#gzip  on;</span><span>server</span> {
        <span>listen</span><span>80</span>;
        <span>server_name</span>  localhost;

        <span>#charset koi8-r;</span><span>#access_log  logs/host.access.log  main;</span><span>location</span> / {
            <span>proxy_set_header</span> Host <span>$host</span>;
        <span>proxy_set_header</span> X-Forward-For <span>$remote_addr</span>;
        <span>proxy_pass</span><span>http://my_server_pool</span>;
       <span># root   html;</span><span>#index  index.html index.htm;</span>
        }
    <span>location</span><span>~ .*\.(gif|jpg|jpeg|png|bmp|swf|js|css)$</span>
    {
        <span>#使用web缓存区cache_one</span><span>proxy_cache</span> cache_one;

        <span>#对不同状态码设置不同缓存时间</span><span>proxy_cache_valid</span><span>200</span><span>304</span><span>12h</span>;
        <span>proxy_cache_valid</span><span>301</span><span>302</span><span>1m</span>;
        <span>proxy_cache_valid</span> any im;
        <span>#设置web缓存的key值，nginx根据key值md5希哈存储缓存，这里根据“域名/URL 参数”组合成key。</span><span>proxy_cache_key</span><span>$host</span><span>$uri</span><span>$is_args</span><span>$args</span>;
        <span>#反向代理，访问后端内容源服务器</span><span>proxy_set_header</span> Host <span>$host</span>;
        <span>proxy_set_header</span> X-Forwarded-For <span>$remote_addr</span>;
        <span>proxy_pass</span> http:my_server_pool;
    }
    <span>#用于清除缓存，假设一个URL为http://my.domain.com/text.gif通过访问http://my.domain.com/purge/test.gif可以清除该URK缓存。</span><span>location</span><span>~ /purge(/.*)</span>
    {
        <span>#设定只允许指定的IP或IP段才可以清除URL缓存。</span><span>allow</span><span>127.0.0.1</span>
        allow       <span>192.168.0.0</span>/<span>16</span>;
        <span>deny</span>        all;
        <span>proxy_cache_purge</span> cache_one <span>$shot</span><span>$1</span><span>$is</span>-args<span>$args</span>;
    }
    <span>access_log</span> 0ff

        <span>#error_page  404              /404.html;</span><span># redirect server error pages to the static page /50x.html</span><span>#</span>
        error_page   <span>500</span><span>502</span><span>503</span><span>504</span>  /50x.html;
        <span>location</span> = /50x.html {
            <span>root</span>   html;
        }

        <span># proxy the PHP scripts to Apache listening on 127.0.0.1:80</span><span>#</span><span>#location ~ \.php$ {</span><span>#    proxy_pass   http://127.0.0.1;</span><span>#}</span><span># pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000</span><span>#</span><span>#location ~ \.php$ {</span><span>#    root           html;</span><span>#    fastcgi_pass   127.0.0.1:9000;</span><span>#    fastcgi_index  index.php;</span><span>#    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;</span><span>#    include        fastcgi_params;</span><span>#}</span><span># deny access to .htaccess files, if Apache's document root</span><span># concurs with nginx's one</span><span>#</span><span>#location ~ /\.ht {</span><span>#    deny  all;</span><span>#}</span>
    }


    <span># another virtual host using mix of IP-, name-, and port-based configuration</span><span>#</span><span>#server {</span><span>#    listen       8000;</span><span>#    listen       somename:8080;</span><span>#    server_name  somename  alias  another.alias;</span><span>#    location / {</span><span>#        root   html;</span><span>#        index  index.html index.htm;</span><span>#    }</span><span>#}</span><span># HTTPS server</span><span>#</span><span>#server {</span><span>#    listen       443 ssl;</span><span>#    server_name  localhost;</span><span>#    ssl_certificate      cert.pem;</span><span>#    ssl_certificate_key  cert.key;</span><span>#    ssl_session_cache    shared:SSL:1m;</span><span>#    ssl_session_timeout  5m;</span><span>#    ssl_ciphers  HIGH:!aNULL:!MD5;</span><span>#    ssl_prefer_server_ciphers  on;</span><span>#    location / {</span><span>#        root   html;</span><span>#        index  index.html index.htm;</span><span>#    }</span><span>#}</span>}