- Egress Gateways with TLS Origination
- Before you begin
- Perform TLS origination with an egress gateway
- Cleanup the TLS origination example
- Perform mutual TLS origination with an egress gateway
- Generate client and server certificates and keys
- Deploy a mutual TLS server
- Deploy a container to test the NGINX deployment
- Redeploy the Egress Gateway with the client certificates
- Configure mutual TLS origination for egress traffic
- Cleanup the mutual TLS origination example
- Cleanup
- 相关内容
Egress Gateways with TLS Origination
The TLS Origination for Egress Trafficexample shows how to configure Istio to perform TLS originationfor traffic to an external service. The Configure an Egress Gatewayexample shows how to configure Istio to direct egress traffic through adedicated egress gateway service. This example combines the previous two bydescribing how to configure an egress gateway to perform TLS origination fortraffic to external services.
Before you begin
Setup Istio by following the instructions in the Installation guide.
Start the sleep samplewhich will be used as a test source for external calls.
If you have enabled automatic sidecar injection, do
Zip
$ kubectl apply -f @samples/sleep/sleep.yaml@
otherwise, you have to manually inject the sidecar before deploying the sleep
application:
Zip
$ kubectl apply -f <(istioctl kube-inject -f @samples/sleep/sleep.yaml@)
Note that any pod that you can exec
and curl
from would do.
- Create a shell variable to hold the name of the source pod for sending requests to external services.If you used the sleep sample, run:
$ export SOURCE_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})
Deploy Istio egress gateway.
Enable Envoy’s access logging
Perform TLS origination with an egress gateway
This section describes how to perform the same TLS origination as in theTLS Origination for Egress Traffic example,only this time using an egress gateway. Note that in this case the TLS origination willbe done by the egress gateway, as opposed to by the sidecar in the previous example.
- Define a
ServiceEntry
foredition.cnn.com
:
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: cnn
spec:
hosts:
- edition.cnn.com
ports:
- number: 80
name: http
protocol: HTTP
- number: 443
name: https
protocol: HTTPS
resolution: DNS
EOF
- Verify that your
ServiceEntry
was applied correctly by sending a request to http://edition.cnn.com/politics.
$ kubectl exec -it $SOURCE_POD -c sleep -- curl -sL -o /dev/null -D - http://edition.cnn.com/politics
HTTP/1.1 301 Moved Permanently
...
location: https://edition.cnn.com/politics
...
command terminated with exit code 35
Your ServiceEntry
was configured correctly if you see 301 Moved Permanently in the output.
- Create an egress
Gateway
for edition.cnn.com, port 443, and a destination rule forsidecar requests that will be directed to the egress gateway.
Choose the instructions corresponding to whether or not you want to enablemutual TLS Authentication between the source pod and the egress gateway.
You may want to enable mutual TLS so the traffic between the source pod and the egress gateway will be encrypted.In addition, mutual TLS will allow the egress gateway to monitor the identity of the source pods and enable Mixerpolicy enforcement based on that identity.
mutual TLS enabledmutual TLS disabled
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- port:
number: 80
name: https
protocol: HTTPS
hosts:
- edition.cnn.com
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-cnn
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: cnn
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 80
tls:
mode: ISTIO_MUTUAL
sni: edition.cnn.com
EOF
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- port:
number: 80
name: http-port-for-tls-origination
protocol: HTTP
hosts:
- edition.cnn.com
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-cnn
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: cnn
EOF
- Define a
VirtualService
to direct the traffic through the egress gateway, and aDestinationRule
to perform TLS origination for requests toedition.cnn.com
:
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-cnn-through-egress-gateway
spec:
hosts:
- edition.cnn.com
gateways:
- istio-egressgateway
- mesh
http:
- match:
- gateways:
- mesh
port: 80
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: cnn
port:
number: 80
weight: 100
- match:
- gateways:
- istio-egressgateway
port: 80
route:
- destination:
host: edition.cnn.com
port:
number: 443
weight: 100
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: originate-tls-for-edition-cnn-com
spec:
host: edition.cnn.com
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: SIMPLE # initiates HTTPS for connections to edition.cnn.com
EOF
- Send an HTTP request to http://edition.cnn.com/politics.
$ kubectl exec -it $SOURCE_POD -c sleep -- curl -sL -o /dev/null -D - http://edition.cnn.com/politics
HTTP/1.1 200 OK
...
content-length: 150793
...
The output should be the same as in the TLS Origination for Egress Trafficexample, with TLS origination: without the 301 Moved Permanently message.
- Check the log of the
istio-egressgateway
pod and you should see a line corresponding to our request.If Istio is deployed in theistio-system
namespace, the command to print the log is:
$ kubectl logs -l istio=egressgateway -c istio-proxy -n istio-system | tail
You should see a line similar to the following:
"[2018-06-14T13:49:36.340Z] "GET /politics HTTP/1.1" 200 - 0 148528 5096 90 "172.30.146.87" "curl/7.35.0" "c6bfdfc3-07ec-9c30-8957-6904230fd037" "edition.cnn.com" "151.101.65.67:443"
Cleanup the TLS origination example
Remove the Istio configuration items you created:
$ kubectl delete gateway istio-egressgateway
$ kubectl delete serviceentry cnn
$ kubectl delete virtualservice direct-cnn-through-egress-gateway
$ kubectl delete destinationrule originate-tls-for-edition-cnn-com
$ kubectl delete destinationrule egressgateway-for-cnn
Perform mutual TLS origination with an egress gateway
Similar to the previous section, this section describes how to configure an egress gateway to performTLS origination for an external service, only this time using a service that requires mutual TLS.
This example is considerably more involved because you need to first:
- generate client and server certificates
- deploy an external service that supports the mutual TLS protocol
- redeploy the egress gateway with the needed mutual TLS certsOnly then can you configure the external traffic to go through the egress gateway which will performTLS origination.
Generate client and server certificates and keys
- Clone the https://github.com/nicholasjackson/mtls-go-example repository:
$ git clone https://github.com/nicholasjackson/mtls-go-example
- Change directory to the cloned repository:
$ cd mtls-go-example
- Generate the certificates for
nginx.example.com
.Use any password with the following command:
$ ./generate.sh nginx.example.com <password>
Select y
for all prompts that appear.
- Move the certificates into the
nginx.example.com
directory:
$ mkdir ../nginx.example.com && mv 1_root 2_intermediate 3_application 4_client ../nginx.example.com
- Go back to your previous directory:
$ cd ..
Deploy a mutual TLS server
To simulate an actual external service that supports the mutual TLS protocol,deploy an NGINX server in your Kubernetes cluster, but running outside ofthe Istio service mesh, i.e., in a namespace without Istio sidecar proxy injection enabled.
- Create a namespace to represent services outside the Istio mesh, namely
mesh-external
. Note that the sidecar proxy willnot be automatically injected into the pods in this namespace since the automatic sidecar injection was notenabled on it.
$ kubectl create namespace mesh-external
- Create Kubernetes Secrets to hold the server’s and CAcertificates.
$ kubectl create -n mesh-external secret tls nginx-server-certs --key nginx.example.com/3_application/private/nginx.example.com.key.pem --cert nginx.example.com/3_application/certs/nginx.example.com.cert.pem
$ kubectl create -n mesh-external secret generic nginx-ca-certs --from-file=nginx.example.com/2_intermediate/certs/ca-chain.cert.pem
- Create a configuration file for the NGINX server:
$ cat <<EOF > ./nginx.conf
events {
}
http {
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
error_log /var/log/nginx/error.log;
server {
listen 443 ssl;
root /usr/share/nginx/html;
index index.html;
server_name nginx.example.com;
ssl_certificate /etc/nginx-server-certs/tls.crt;
ssl_certificate_key /etc/nginx-server-certs/tls.key;
ssl_client_certificate /etc/nginx-ca-certs/ca-chain.cert.pem;
ssl_verify_client on;
}
}
EOF
- Create a Kubernetes ConfigMapto hold the configuration of the NGINX server:
$ kubectl create configmap nginx-configmap -n mesh-external --from-file=nginx.conf=./nginx.conf
- Deploy the NGINX server:
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
name: my-nginx
namespace: mesh-external
labels:
run: my-nginx
spec:
ports:
- port: 443
protocol: TCP
selector:
run: my-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
namespace: mesh-external
spec:
selector:
matchLabels:
run: my-nginx
replicas: 1
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 443
volumeMounts:
- name: nginx-config
mountPath: /etc/nginx
readOnly: true
- name: nginx-server-certs
mountPath: /etc/nginx-server-certs
readOnly: true
- name: nginx-ca-certs
mountPath: /etc/nginx-ca-certs
readOnly: true
volumes:
- name: nginx-config
configMap:
name: nginx-configmap
- name: nginx-server-certs
secret:
secretName: nginx-server-certs
- name: nginx-ca-certs
secret:
secretName: nginx-ca-certs
EOF
- Define a
ServiceEntry
and aVirtualService
fornginx.example.com
to instruct Istio to direct traffic destinedtonginx.example.com
to your NGINX server:
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: nginx
spec:
hosts:
- nginx.example.com
ports:
- number: 80
name: http
protocol: HTTP
- number: 443
name: https
protocol: HTTPS
resolution: DNS
endpoints:
- address: my-nginx.mesh-external.svc.cluster.local
ports:
https: 443
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: nginx
spec:
hosts:
- nginx.example.com
tls:
- match:
- port: 443
sni_hosts:
- nginx.example.com
route:
- destination:
host: nginx.example.com
port:
number: 443
weight: 100
EOF
Deploy a container to test the NGINX deployment
- Create Kubernetes Secrets to hold the client’s and CAcertificates:
$ kubectl create secret tls nginx-client-certs --key nginx.example.com/4_client/private/nginx.example.com.key.pem --cert nginx.example.com/4_client/certs/nginx.example.com.cert.pem
$ kubectl create secret generic nginx-ca-certs --from-file=nginx.example.com/2_intermediate/certs/ca-chain.cert.pem
- Deploy the sleep sample with mounted client and CA certificates to test sendingrequests to the NGINX server:
$ kubectl apply -f - <<EOF
# Copyright 2017 Istio Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
############################################################
# Sleep service
############################################################
apiVersion: v1
kind: Service
metadata:
name: sleep
labels:
app: sleep
spec:
ports:
- port: 80
name: http
selector:
app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sleep
spec:
replicas: 1
template:
metadata:
labels:
app: sleep
spec:
containers:
- name: sleep
image: tutum/curl
command: ["/bin/sleep","infinity"]
imagePullPolicy: IfNotPresent
volumeMounts:
- name: nginx-client-certs
mountPath: /etc/nginx-client-certs
readOnly: true
- name: nginx-ca-certs
mountPath: /etc/nginx-ca-certs
readOnly: true
volumes:
- name: nginx-client-certs
secret:
secretName: nginx-client-certs
- name: nginx-ca-certs
secret:
secretName: nginx-ca-certs
EOF
- Define an environment variable to hold the name of the
sleep
pod:
$ export SOURCE_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})
- Use the deployed sleep pod to send requests to the NGINX server.Since
nginx.example.com
does not actually exist and therefore DNS cannot resolve it, the followingcurl
command uses the—resolve
option to resolve the hostname manually. The IP value passed in the–resolve option (1.1.1.1 below) is not significant. Any value other than 127.0.0.1 can be used.Normally, a DNS entry exists for the destination hostname and you would not use the—resolve
option ofcurl
.
$ kubectl exec -it $SOURCE_POD -c sleep -- curl -v --resolve nginx.example.com:443:1.1.1.1 --cacert /etc/nginx-ca-certs/ca-chain.cert.pem --cert /etc/nginx-client-certs/tls.crt --key /etc/nginx-client-certs/tls.key https://nginx.example.com
...
Server certificate:
subject: C=US; ST=Denial; L=Springfield; O=Dis; CN=nginx.example.com
start date: 2018-08-16 04:31:20 GMT
expire date: 2019-08-26 04:31:20 GMT
common name: nginx.example.com (matched)
issuer: C=US; ST=Denial; O=Dis; CN=nginx.example.com
SSL certificate verify ok.
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: nginx.example.com
...
< HTTP/1.1 200 OK
< Server: nginx/1.15.2
...
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
- Verify that the server requires the client’s certificate:
$ kubectl exec -it $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name}) -c sleep -- curl -k --resolve nginx.example.com:443:1.1.1.1 https://nginx.example.com
<html>
<head><title>400 No required SSL certificate was sent</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<center>No required SSL certificate was sent</center>
<hr><center>nginx/1.15.2</center>
</body>
</html>
Redeploy the Egress Gateway with the client certificates
- Create Kubernetes Secrets to hold the client’s and CAcertificates.
$ kubectl create -n istio-system secret tls nginx-client-certs --key nginx.example.com/4_client/private/nginx.example.com.key.pem --cert nginx.example.com/4_client/certs/nginx.example.com.cert.pem
$ kubectl create -n istio-system secret generic nginx-ca-certs --from-file=nginx.example.com/2_intermediate/certs/ca-chain.cert.pem
- Generate the
istio-egressgateway
deployment with a volume to be mounted from the new secrets. Use the same optionsyou used for generating youristio.yaml
:
$ istioctl manifest generate --set values.gateways.istio-ingressgateway.enabled=false \
--set values.gateways.istio-egressgateway.enabled=true \
--set 'values.gateways.istio-egressgateway.secretVolumes[0].name'=egressgateway-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[0].secretName'=istio-egressgateway-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[0].mountPath'=/etc/istio/egressgateway-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[1].name'=egressgateway-ca-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[1].secretName'=istio-egressgateway-ca-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[1].mountPath'=/etc/istio/egressgateway-ca-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[2].name'=nginx-client-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[2].secretName'=nginx-client-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[2].mountPath'=/etc/nginx-client-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[3].name'=nginx-ca-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[3].secretName'=nginx-ca-certs \
--set 'values.gateways.istio-egressgateway.secretVolumes[3].mountPath'=/etc/nginx-ca-certs > \
./istio-egressgateway.yaml
- Redeploy
istio-egressgateway
:
$ kubectl apply -f ./istio-egressgateway.yaml
deployment "istio-egressgateway" configured
- Verify that the key and the certificate are successfully loaded in the
istio-egressgateway
pod:
$ kubectl exec -it -n istio-system $(kubectl -n istio-system get pods -l istio=egressgateway -o jsonpath='{.items[0].metadata.name}') -- ls -al /etc/nginx-client-certs /etc/nginx-ca-certs
tls.crt
and tls.key
should exist in /etc/istio/nginx-client-certs
, while ca-chain.cert.pem
in/etc/istio/nginx-ca-certs
.
Configure mutual TLS origination for egress traffic
- Create an egress
Gateway
fornginx.example.com
, port 443, and destination rules andvirtual services to direct the traffic through the egress gateway and from the egress gateway to the externalservice.
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- nginx.example.com
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-nginx
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: nginx
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: ISTIO_MUTUAL
sni: nginx.example.com
EOF
- Define a
VirtualService
to direct the traffic through the egress gateway, and aDestinationRule
to performmutual TLS origination:
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-nginx-through-egress-gateway
spec:
hosts:
- nginx.example.com
gateways:
- istio-egressgateway
- mesh
http:
- match:
- gateways:
- mesh
port: 80
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: nginx
port:
number: 443
weight: 100
- match:
- gateways:
- istio-egressgateway
port: 443
route:
- destination:
host: nginx.example.com
port:
number: 443
weight: 100
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: originate-mtls-for-nginx
spec:
host: nginx.example.com
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: MUTUAL
clientCertificate: /etc/nginx-client-certs/tls.crt
privateKey: /etc/nginx-client-certs/tls.key
caCertificates: /etc/nginx-ca-certs/ca-chain.cert.pem
sni: nginx.example.com
EOF
- Send an HTTP request to
http://nginx.example.com
:
$ kubectl exec -it $SOURCE_POD -c sleep -- curl -s --resolve nginx.example.com:80:1.1.1.1 http://nginx.example.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
- Check the log of the
istio-egressgateway
pod for a line corresponding to our request.If Istio is deployed in theistio-system
namespace, the command to print the log is:
$ kubectl logs -l istio=egressgateway -n istio-system | grep 'nginx.example.com' | grep HTTP
You should see a line similar to the following:
[2018-08-19T18:20:40.096Z] "GET / HTTP/1.1" 200 - 0 612 7 5 "172.30.146.114" "curl/7.35.0" "b942b587-fac2-9756-8ec6-303561356204" "nginx.example.com" "172.21.72.197:443"
Cleanup the mutual TLS origination example
- Remove created Kubernetes resources:
$ kubectl delete secret nginx-server-certs nginx-ca-certs -n mesh-external
$ kubectl delete secret nginx-client-certs nginx-ca-certs
$ kubectl delete secret nginx-client-certs nginx-ca-certs -n istio-system
$ kubectl delete configmap nginx-configmap -n mesh-external
$ kubectl delete service my-nginx -n mesh-external
$ kubectl delete deployment my-nginx -n mesh-external
$ kubectl delete namespace mesh-external
$ kubectl delete gateway istio-egressgateway
$ kubectl delete serviceentry nginx
$ kubectl delete virtualservice direct-nginx-through-egress-gateway
$ kubectl delete destinationrule originate-mtls-for-nginx
$ kubectl delete destinationrule egressgateway-for-nginx
- Delete the directory of certificates and the repository used to generate them:
$ rm -rf nginx.example.com mtls-go-example
- Delete the generated configuration files used in this example:
$ rm -f ./nginx.conf ./istio-egressgateway.yaml
Cleanup
Delete the sleep
service and deployment:
$ kubectl delete service sleep
$ kubectl delete deployment sleep
相关内容
Secure Control of Egress Traffic in Istio, part 3
Comparison of alternative solutions to control egress traffic including performance considerations.
Secure Control of Egress Traffic in Istio, part 2
Use Istio Egress Traffic Control to prevent attacks involving egress traffic.
Secure Control of Egress Traffic in Istio, part 1
Attacks involving egress traffic and requirements for egress traffic control.
Egress Gateway Performance Investigation
Verifies the performance impact of adding an egress gateway.
Consuming External MongoDB Services
Describes a simple scenario based on Istio's Bookinfo example.
Monitoring and Access Policies for HTTP Egress Traffic
Describes how to configure Istio for monitoring and access policies of HTTP egress traffic.