How to route mssql traffic through an Istio egress gateway - sql-server

Question

I am trying to run a .Net 6 demonstration application in an Istio service mesh (Istio 1.16.1 in an AKS cluster). This application uses a sqlserver 2019 located outside the cluster and I would like to route all outgoing traffic, including mssql, through an egress gateway.
Please note this application also use OpenId Connect and use keytabs (Kerberos traffic), I have successfully managed to route those requests through the egress gateway but not the mssql traffic.
I have created the service mesh with istioctl and the following configuration file
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
profile: default
meshConfig:
accessLogFile: /dev/stdout
enableTracing: true
defaultConfig:
tracing:
sampling: 100
outboundTrafficPolicy:
mode: REGISTRY_ONLY
components:
pilot:
k8s:
nodeSelector:
agentpool: svcmaster
ingressGateways:
- name: istio-ingressgateway
enabled: true
k8s:
nodeSelector:
kubernetes.io/os: linux
egressGateways:
- name: istio-egressgateway
enabled: true
k8s:
nodeSelector:
kubernetes.io/os: linux
Here is the ServiceEntry for the database
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: mssql-contoso-com
namespace: linux
spec:
hosts:
- mssql.contoso.com
addresses:
- 10.1.0.5
ports:
- number: 1433
name: mssql
protocol: TLS
- number: 443
name: tls
protocol: TLS
location: MESH_EXTERNAL
resolution: DNS
Here is the gateway (it includes the host
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: egress-gateway
namespace: linux
spec:
selector:
istio: egressgateway
servers:
- port:
number: 443
name: tls
protocol: TLS
hosts:
- "adfs.contoso.com"
- "mssql.contoso.com"
tls:
mode: "PASSTHROUGH"
- port:
number: 80
name: tcp
protocol: TCP
hosts:
- "controller.contoso.com"
And finally, the VirtualService. I have not defined a DestinationRule because it is actually useless, the OIDC and Kerberos traffic are routed correctly without them and I have tried to add it out of desperation without solving the issue.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: "outgoing-mssql-traffic"
namespace: linux
spec:
hosts:
- mssql.contoso.com
gateways:
- egress-gateway
- mesh
tls:
- match:
- gateways:
- mesh
port: 1433
sniHosts:
- mssql.contoso.com
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
port:
number: 443
weight: 100
- match:
- gateways:
- egress-gateway
port: 443
sniHosts:
- mssql.contoso.com
route:
- destination:
host: mssql.contoso.com
port:
number: 1433
weight: 100
Regarding the details of the application call to the SQL Server, I am using a regular SQLConnection with the following connection string:
Server=mssql.contoso.com;Initial Catalog=Demonstration;Integrated Security=true;TrustServerCertificate=true
As a result, I get the following error in the application log:
Microsoft.Data.SqlClient.SqlException (0x80131904): A connection was successfully established with the server, but then an error occurred during the pre-login handshake. (provider: TCP Provider, error: 35 - An internal exception was caught)
---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
---> System.Net.Sockets.SocketException (104): Connection reset by peer
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 count)
Somehow the TLS handshake fails. When consulting the logs of the sidecar container and of the egress gateway, I cannot see the traffic to the database. I have also monitored the traffic on the SQLServer machine with Wireshark and I cannot see TCP traffic on port 1433.
The application works fine when the virtual service is deleted so the issue is really related to the routing through the egress gateway.
Any help or insight would be appreciated.

Answer 1

So I found out the reason a bit by accident while trying different configurations and checking the Envoy logs. The Istio Egress gateway service exposes only two ports (at least by default): 80 and 443. It seems that each port can only serve one VirtualService and it is first come first served. I had declared the VirtualService for mssql last, so it was not served.
Furthermore, I had to switch to a TCP route. I failed to get it working with a TLS route.
Here is the ServiceEntry:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: mssql-contoso-com
namespace: linux
spec:
hosts:
- mssql.contoso.com
addresses:
- 10.1.0.5
ports:
- number: 1433
name: mssql
protocol: TCP
- number: 80
name: tls
protocol: TCP
location: MESH_EXTERNAL
resolution: DNS
And here is the VirtualService:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: "outgoing-mssql-traffic"
namespace: linux
spec:
hosts:
- mssql.contoso.com
gateways:
- egress-gateway
- mesh
tcp:
- match:
- gateways:
- mesh
port: 1433
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
port:
number: 80
weight: 100
- match:
- gateways:
- egress-gateway
port: 80
route:
- destination:
host: mssql.contoso.com
port:
number: 1433
weight: 100
I guess that I will have to either deploy several egress gateways or find a way to add ports to the existing one.