2016年7月，中國(guó)銀監(jiān)會(huì)發(fā)布了《中國(guó)銀行業(yè)信息科技“十三五”發(fā)展規(guī)劃監(jiān)管指導(dǎo)意見(jiàn)（征求意見(jiàn)稿）》（下稱(chēng)《意見(jiàn)》），《意見(jiàn)》表示，探索構(gòu)建私有云平臺(tái)，采用成熟度高、開(kāi)放性強(qiáng)的計(jì)算虛擬化、容器虛擬化、分布式存儲(chǔ)、網(wǎng)絡(luò)虛擬化等技術(shù)，建立資源池，形成資源彈性供給、靈活調(diào)度和動(dòng)態(tài)計(jì)量的私有云平臺(tái)。同時(shí)，同步開(kāi)展應(yīng)用架構(gòu)規(guī)劃，構(gòu)建與云計(jì)算基礎(chǔ)設(shè)施相適應(yīng)的應(yīng)用架構(gòu)，自主設(shè)計(jì)或推動(dòng)應(yīng)用開(kāi)發(fā)商實(shí)施應(yīng)用架構(gòu)改造，并降低應(yīng)用與基礎(chǔ)架構(gòu)的耦合度。從《意見(jiàn)》的描述可以看到，容器技術(shù)天生的優(yōu)勢(shì)以及在大規(guī)模集群管理、混合云平臺(tái)構(gòu)建、微服務(wù)化應(yīng)用以及云原生應(yīng)用等場(chǎng)景的廣泛應(yīng)用，很大程度上與金融業(yè)IT 的戰(zhàn)略轉(zhuǎn)型發(fā)展規(guī)劃相契合。容器技術(shù)正在為傳統(tǒng)金融業(yè)IT 技術(shù)轉(zhuǎn)型提供一種顛覆性的轉(zhuǎn)型思路。
傳統(tǒng)WAS容器化及相關(guān)實(shí)踐
從2016開(kāi)始，睿云智合(Wise2C)接觸了一定數(shù)量的金融客戶(hù)同時(shí)實(shí)施了好幾個(gè)容器化管理平臺(tái)的項(xiàng)目，項(xiàng)目中實(shí)施的中間件包括Websphere/WebLogic/JBOSS EAP /TOMCAT等，在項(xiàng)目中針對(duì)不同客戶(hù)的具體應(yīng)用的容器化感受頗深，理論上重型應(yīng)用服務(wù)器Websphere Application Server(簡(jiǎn)稱(chēng)WAS)和WebLogic不能很好的在容器云平臺(tái)充分發(fā)揮大規(guī)模部署和彈性伸縮的優(yōu)勢(shì)，但是在當(dāng)前金融客戶(hù)實(shí)際環(huán)境中，大量的應(yīng)用仍然按照傳統(tǒng)的方式部署在WAS和WebLogic上，所以睿云智合(Wise2C)也積極做了相關(guān)研究和探索以支持傳統(tǒng)的應(yīng)用服務(wù)器WAS和WebLogic容器化，滿(mǎn)足當(dāng)前金融客戶(hù)的需求，此篇文章主要介紹傳統(tǒng)WAS容器化及相關(guān)實(shí)踐。
首先對(duì)典型的應(yīng)用服務(wù)器的特性做個(gè)對(duì)比，參考下圖，由此我們不難知道為什么WAS在金融行業(yè)仍然占主導(dǎo)地位。
這里就不對(duì)WAS的基本概念做展開(kāi)了，當(dāng)前金融客戶(hù)中WAS應(yīng)用服務(wù)服務(wù)器運(yùn)行主要包括兩種模式：Standalone和ND集群。在之前的項(xiàng)目中我們已經(jīng)對(duì)Nginx+多節(jié)點(diǎn)WAS Standalone 和WAS ND Cluster+HIS分別進(jìn)行了容器化并進(jìn)行了相關(guān)簡(jiǎn)單壓測(cè)對(duì)比，并實(shí)現(xiàn)了一鍵部署和彈性伸縮，另配合公司產(chǎn)品WiseBuild實(shí)現(xiàn)Java項(xiàng)目在WAS Standalone持續(xù)集成和持續(xù)部署。
WAS Standalone
上圖是WAS Standalone結(jié)合IBM HTTP Server簡(jiǎn)稱(chēng)IHS，IBM開(kāi)發(fā)了Websphere Plug-in以實(shí)現(xiàn)IHS和WAS的緊密結(jié)合，實(shí)測(cè)中IHS性能表現(xiàn)優(yōu)秀，但是Nginx作為當(dāng)前另一款優(yōu)秀的Web Server,我們參考相關(guān)官方資料用它取代IHS并結(jié)合rancher-gen實(shí)現(xiàn)了WAS在Rancher容器管理平臺(tái)的一鍵部署和彈性伸縮，請(qǐng)參考下圖
金融行業(yè)當(dāng)前主流WAS應(yīng)用服務(wù)器的版本是8，與舊版本相比,IBM已經(jīng)用Installation Manager取代之前的updateinstaller,在WAS的dockerlized容器化中我們需要用靜默安裝取代GUI的圖形化安裝，以下是WAS安裝的Dockerfile
WAS安裝的Dockerfile
FROM rhel:7.2
ARG USER=wasdmin
ARG GROUP=wasadmin
COPY scripts /scripts
ARG URL=http://harbor01.test01.com:8080/was8/
ENV PATH /websphere/IBM/WebSphere/AppServer/bin:$PATH
###################### IBM Installation Manager ##########################
?
# Install IBM Installation Manager
RUN groupadd $GROUP && useradd -g $GROUP $USER -m \
????&& mkdir -p /websphere/IBM/WebSphere/AppServer/ \
????&& chown -R $USER.$GROUP ?/websphere /tmp /var \
????&& ?wget -q $URL/agent.installer.linux.gtk.x86_64_1.6.2000.20130301_2248.zip -O /tmp/IM.zip \
????&& mkdir /tmp/im && ?unzip -qd /tmp/im /tmp/IM.zip \
????&& /tmp/im/installc -acceptLicense -accessRights nonAdmin \
??????-installationDirectory “/websphere/IBM/InstallationManager” ?\
??????-dataLocation “/var/ibm/InstallationManager” -showProgress \
????&& rm -fr /tmp/IM.zip /tmp/im
?
################# IBM WebSphere Application Server ######################
?
# Install IBM WebSphere Application Server v855
RUN wget -q $URL/WASND_v8.5.5_1of3.zip -O /tmp/was1.zip \
????&& wget -q $URL/WASND_v8.5.5_2of3.zip -O /tmp/was2.zip\
????&& wget -q $URL/WASND_v8.5.5_3of3.zip -O /tmp/was3.zip \
????&& mkdir /tmp/was && unzip -qd /tmp/was /tmp/was1.zip \
????&& unzip -qd /tmp/was /tmp/was2.zip \
????&& unzip -qd /tmp/was /tmp/was3.zip \
????&& /websphere/IBM/InstallationManager/eclipse/tools/imcl -showProgress \
??????-acceptLicense install com.ibm.websphere.ND.v85_8.5.5000.20130514_1044 \
??????-repositories /tmp/was/repository.config ?\
??????-installationDirectory /websphere/IBM/WebSphere/AppServer \
??????-preferences com.ibm.cic.common.core.preferences.preserveDownloadedArtifacts=false \
????&& rm -fr /tmp/was /tmp/was1.zip /tmp/was2.zip /tmp/was3.zip
?
############## IBM WebSphere Application Server Fixpack ##################
?
# Install IBM WebSphere Application Server Fixpack v85510
RUN wget -q $URL/8.5.5-WS-WAS-FP0000009-part1.zip -O /tmp/wasfp1.zip \
????&& wget -q $URL/8.5.5-WS-WAS-FP0000009-part2.zip -O /tmp/wasfp2.zip \
????&& mkdir /tmp/wasfp \
????&& unzip -qd /tmp/wasfp /tmp/wasfp1.zip ?\
????&& unzip -qd /tmp/wasfp /tmp/wasfp2.zip \
????&& /websphere/IBM/InstallationManager/eclipse/tools/imcl -showProgress \
??????-acceptLicense install com.ibm.websphere.ND.v85_8.5.5009.20160225_0435 \
??????-repositories /tmp/wasfp/repository.config ?\
??????-installationDirectory /websphere/IBM/WebSphere/AppServer \
??????-preferences com.ibm.cic.common.core.preferences.preserveDownloadedArtifacts=false \
????&& rm -fr /tmp/wasfp /tmp/wasfp1.zip /tmp/wasfp2.zip
?
########################### Install Java SDK 8.0 ########################
?
# Install Java SDK 8.0
RUN ?wget -q $URL/7.1.3.30-WS-IBMWASJAVA-Linux.zip -O /tmp/java.zip \
????&& mkdir /tmp/java \
????&& unzip -qd /tmp/java /tmp/java.zip ?\
????&& /websphere/IBM/InstallationManager/eclipse/tools/imcl -showProgress \
??????-acceptLicense install com.ibm.websphere.IBMJAVA.v71_7.1.3030.20160224_1952 \
??????-repositories /tmp/java/repository.config \
??????-installationDirectory /websphere/IBM/WebSphere/AppServer \
??????-preferences com.ibm.cic.common.core.preferences.preserveDownloadedArtifacts=false \
????&& rm -fr /tmp/java /tmp/java.zip
?
ENV PATH /websphere/IBM/WebSphere/AppServer/bin:$PATH
RUN managesdk.sh -setCommandDefault -sdkname 1.7.1_64 \
????&& managesdk.sh -setNewProfileDefault -sdkname 1.7.1_64 \
????&& /websphere/IBM/WebSphere/AppServer/bin/manageprofiles.sh -create -profileName AppSrv01 \
???????-profilePath /websphere/IBM/WebSphere/AppServer/profiles/AppSrv01 ?\
???????-templatePath /websphere/IBM/WebSphere/AppServer/profileTemplates/default \
???????-nodeName DefaultNode01 -cellName DefaultCell01 -hostName localhost \
???????-serverName server1 -enableAdminSecurity true -adminUserName wasadmin -adminPassword “wasadmin” \
????&& chown -R $USER.$GROUP ?/websphere
CMD [“tar”,”cvf”,”/tmp/was.tar”,”/websphere/IBM/WebSphere/AppServer”]
構(gòu)建基礎(chǔ)WAS安裝鏡像
docker build –build-arg USER=<user> –build-arg GROUP=<group> \
–build-arg URL=<URL> -t ibmwas8:prereq ?.
這里如果不指定參數(shù)將使用Dockerfile的默認(rèn)參數(shù)，生成鏡像ibmwas8:prereq 然后基于此鏡像運(yùn)行下面指令產(chǎn)生一個(gè)完整的WAS安裝tar包：was.tar.
docker run –rm -v $(pwd):/tmp ?ibmwas8:prereq
有了基本的WAS安裝包，就可以分別構(gòu)建WAS Standalone 和WAS ND Cluster,當(dāng)然構(gòu)建后者是相當(dāng)復(fù)雜的，這里主要介紹下構(gòu)建WAS Standalone和Nginx結(jié)合
以下是構(gòu)建WAS Standalone的Dockerfile
FROM rhel:7.2
?
ARG USER=wasadmin
ARG GROUP=wasadmin
?
COPY scripts /scripts
RUN mkdir -p /websphere/ \
????&& groupadd $GROUP \
????&& useradd $USER -g $GROUP -m \
????&& chown -R $USER.$GROUP /scripts \
????&& chmod +x /scripts/*
?
#USER $USER
ADD was.tar /websphere/
RUN /scripts/create_profile.sh
ENV PATH ?/websphere/IBM/WebSphere/AppServer/bin:$PATH
EXPOSE 9080 9043
CMD [ “/scripts/startserver.sh” ]
其中需要注意的是如果想WAS Standalone能被容器快速拉起的話(huà)，需要把創(chuàng)建profile直接打入鏡像，否則通過(guò)ENTRYPOINT或者CMD在容器拉起時(shí)執(zhí)行創(chuàng)建profile將會(huì)比較慢
Rancher-Gen,結(jié)合Nginx
Rancher-Gen是一個(gè)Python的工具，它能監(jiān)聽(tīng)的Rancher服務(wù)的變化，并呈現(xiàn)一個(gè)用戶(hù)指定的Jinja2模板。這允許用戶(hù)為現(xiàn)有的、并基于這些變化的服務(wù)生成配置文件。另外，Rancher-Gen提供了一種機(jī)制，在模板呈現(xiàn)之后會(huì)運(yùn)行通知命令,基于此，所以它能自動(dòng)檢測(cè)后端服務(wù)的變化，并實(shí)現(xiàn)nginx的動(dòng)態(tài)更新
構(gòu)建Nginx和Rancher-Gen的Dockerfile
FROM phusion/baseimage
MAINTAINER pitrho
# Step 1– Install nginx and python
ENV DEBIAN_FRONTEND noninteractive
RUN \
?apt-add-repository -y ppa:nginx/stable && \
?apt-get update && \
?apt-get install -y python-software-properties \
???wget \
???nginx \
???python-dev \
???python-pip \
???libev4 \
???libev-dev \
???expect-dev && \
?rm -rf /var/lib/apt/lists/* && \
?chown -R www-data:www-data /var/lib/nginx && \
apt-get clean
?
# Step 2?– Install rancher-gen
ENV RANCHER_GEN_VERSION 0.3.0
RUN pip install rancher-gen==$RANCHER_GEN_VERSION
?
# Step 3?– Define services
RUN mkdir /etc/service/nginx /etc/service/rancher_gen /nginxconf
COPY nginx_run /etc/service/nginx/run
COPY rancher-gen_run /etc/service/rancher_gen/run
COPY default.j2 /nginxconf
COPY start.sh /
?
# Step 4?– Use baseimage-docker’s init system.
CMD [“/start.sh”]
?
# Step 5?– Expose ports.
EXPOSE 80
EXPOSE 443
?
注意Step3中的rancher-gen_run，正是此條指令啟用了rancher-gen偵聽(tīng)了Rancher的后端服務(wù)，看下腳本內(nèi)容
#!/bin/bash
rancher-gen –host $RANCHER_GEN_HOST –port $RANCHER_GEN_PORT –access-key $RANCHER_GEN_ACCESS_KEY –secret-key $RANCHER_GEN_SECRET_KEY –project-id $RANCHER_GEN_PROJECT_ID $RANCHER_GEN_OPTIONS –notify “service nginx reload” /nginxconf/default.j2 /etc/nginx/sites-available/default
注意在通知步驟之后，我們是怎樣通過(guò)名為?/nginxconf/default.j2 和/etc/nginx/sites-available/default 的這兩個(gè)路徑的。前者是Jinjia2模板，后者是渲染模板的輸出位置
以下是啟用Cache的Nginx配置Jinjia2模板
upstream webserver.backend {
{% for container in containers %}
??{% if container[‘state’] == “running” %}
??server {{container[‘primaryIpAddress’]}}:9080;
??{% endif %}
{% endfor %}
}
?
server {
????listen 80;
????server_name webserver_demo;
?
????location / {
????????proxy_set_header X-Real-IP $remote_addr;
????????proxy_set_header HOST $http_host;
????????proxy_set_header X-NginX-Proxy true;
????????proxy_cache my_cache;
????????proxy_cache_revalidate on;
????????proxy_cache_min_uses 3;
????????proxy_cache_valid any 30m;
????????proxy_pass http://webserver.backend;
????????proxy_redirect off;
????}
}
關(guān)于Rancher-Gen基本介紹就到這里，如果感興趣，請(qǐng)關(guān)注代碼實(shí)現(xiàn)? https://github.com/pitrho/rancher-gen
最后看看怎么樣在Rancher中部署實(shí)現(xiàn)3節(jié)點(diǎn)的WAS Standalone +Nginx的應(yīng)用棧吧
首先編寫(xiě)相應(yīng)的docker-compose.yml和rancher-compose.yml文件
docker.compose.yml
nginx:
??ports:
??– 9080:80/tcp
??environment:
????NGINX_RUN_TYPE: rancher-gen
????RANCHER_GEN_ACCESS_KEY: 560B5CA0B714BEF574E0
????RANCHER_GEN_HOST: 192.168.122.11
????RANCHER_GEN_OPTIONS: –stack WAS-Nginx –service was-standalone
????RANCHER_GEN_PORT: ‘8080’
????RANCHER_GEN_PROJECT_ID: 1a5
????RANCHER_GEN_SECRET_KEY: 7QZqs1TgR412FP5oUL9yyNGivbmnbSc9xqaeDNb5
??labels:
????io.rancher.container.hostname_override: container_name
??tty: true
??image: harbor01.test01.com/base/nginx-rancher-gen-demo
??stdin_open: true
was-standalone:
??labels:
????io.rancher.scheduler.affinity:container_label_soft_ne: io.rancher.stack_service.name=was-nginx/was-standalone
????io.rancher.container.pull_image: always
????io.rancher.container.hostname_override: container_name
??tty: true
??image: harbor01.test01.com/base/ibmwas8:v1
??stdin_open: true
?
rancher-compose.yml
nginx:
scale: 1
was-standalone:
scale: 3
然后切到對(duì)應(yīng)的目錄執(zhí)行rancher-compose up
訪(fǎng)問(wèn)對(duì)應(yīng)的Nginx端口，顯示如下，表示已經(jīng)成功連接到WAS后臺(tái)
到這里文章結(jié)束，希望借此文章拋磚引玉，給大家更多思考!
相關(guān)參考資料
https://developer.ibm.com/wasdev/docs/nginx-websphere-application-server/
https://github.com/WASdev/ci.docker.websphere-traditional
IBM Redbook:sg247957
IBM Redbook:sg248056