When implementing a microservice-based architecture on top of Kubernetes it is always hard to find an ideal alerting strategy, specifically one that ensures reliability during day 2 operations. Modern Kubernetes-based deployments - when built from purely open source components - use Prometheus and the ecosystem built around it for monitoring. The unparalleled scalability of Prometheus allows you to measure (and store) most of the data reported by both workloads and cluster components.

Istio 1.8 has just been released and is one of the best Istio releases so far. The new version contains exciting experimental features, numerous enhancements, as well as deprecations and removals. The core focus of the release, however, is to increase operational stability. In this post, we’ll review what’s new in Istio 1.8, and highlight a few potential snags to look out for when upgrading to the latest version. If you’re looking for a production ready Istio distribution based on the new Istio 1.

Today we’ve launched the 1.5 release of Backyards (now Cisco Service Mesh Manager), Banzai Cloud’s production ready Istio distribution. The vision of Backyards (now Cisco Service Mesh Manager) is to become a go-to solution for SREs and operation engineers to observe and control the health of their services and applications. The service mesh acts as the foundation layer that produces the most important network traffic metrics and traces in a unified way.

We’ve already covered the details of SLO-based alerting for Request Latency (Duration) and Request Error Rate in our previous blog posts. However, if we take a look at Google’s Site Reliability Engineering book, we’ll see that it mentions four golden signals to be used as the basis of any system’s alerting strategy: Latency (or Duration): The time it takes to service a request. Errors: The rate of requests that fail, either explicitly (e.

If you’re an active Istio user, then there’s a good chance that Istio’s configuration reference is bookmarked in your browser, and that you’ve read the pages on VirtualServices, and ServiceEntries over and over, but still have to struggle to set up even simple configurations in your mesh. Istio’s custom resource configuration is very powerful and flexible, but infamous for being overly complex. At its best, its YAML consists of lists of lists, cross-references, conflicting fields, and wildcards.

Hybrid and multi-cloud infrastructure is becoming more and more common in enterprise IT environments, since it provides a superior set of capabilities than what might otherwise be gained from a single cloud. Specifically, it allows us to use different cloud services for different use-cases within a single environment, helping to avoid vendor lock-in. Using both on-prem and cloud environments makes it possible to use the best of both worlds, letting users transition to the cloud at their own pace; it’s very rare for businesses of significant size, nowadays, to exist exclusively on either.

Backyards (now Cisco Service Mesh Manager) provides, out-of-the-box, Service Level Objective-based monitoring and alerting solutions for monitoring HTTP/GRPC-based microservice latencies and error rate characteristics. The decision to only provide these Service Level Indicators as part of the base installation was due to the fact that only Istio itself is able to provide the metrics they require. On the other hand, as we had briefly discussed in the Tracking and enforcing SLOs webinar, the SLO-based measurement of systems can not and should not be limited to these three measurements.

We previously covered the basics of burn rates in our last blog post about enforcing SLOs and in our Tracking and enforcing SLOs webinar. Even if the basic concept is clear (if a burn rate is greater than 1, the SLO is in danger), it can be difficult to develop a strategy for alerts based on burn rate values. Of course, when implemented properly, burn rate-based alerting ensures that Service Level Objectives (SLOs) are enforced and met.

One of the main challenges enterprises currently face – whether they have an on-prem infrastructure or are running their workloads in the cloud – is application and network security. Data must be protected at all times, not just when it’s resting in a database or is placed in storage somewhere. Ensuring data protection while its moving means more than just encryption, it also means authentication and authorization. A service mesh is a transparent infrastructure layer that sits between a network and microservices, as such it’s the perfect place to ensure data encryption, authentication and authorization.

The last few versions of Istio have been released at a rate of about one every three months. Because of this rapid rate the upgrade of an Istio service mesh is an essential and recurring task to do when using Istio. Upgrading an Istio control plane in a production environment with as little downtime as possible has been one of the many challenges that service mesh operators have routinely faced. This is why the new Istio control plane canary upgrade feature is such a game changer.

Today we’ve launched the 1.4 release of Backyards (now Cisco Service Mesh Manager), Banzai Cloud’s production ready Istio distribution. While Backyards (now Cisco Service Mesh Manager) is first and foremost a production ready Istio distribution, the latest release is a shift towards an observability tool and dashboard for SREs and operation teams. The most important features of this release are: designing and tracking service level objectives and corresponding alerting rules seamless canary upgrades of the service mesh control plane support for Istio 1.

Istio 1.7 has just been released and it mostly focuses on improving the operational experience of an Istio service mesh. In this post we’ll review what’s new in Istio 1.7 and highlight a few gotchas to look for when upgrading to this new version. If you’re looking for a production ready Istio distribution based on the new Istio 1.7 release and with support for a safe and automated canary upgrade flow, check out the new Backyards 1.

Running a computing system at scale is hard. The goal of the whole Cloud Native transformation is to make it easier, but it still requires a deep expertise in observability and alerting tools to track the state of the infrastructure and the services. And it’s not only the tools, but the questions that arise at scale. Practice shows that running a complex system with a 100% reliability target is unrealistic. But in that case, how do you determine that certain level of errors that you can live with, or when do you need to trigger alerts to possibly wake an engineer up in the middle of the night?

Companies providing services for government, healthcare, or finance sectors must ensure that their products adhere to government approved security requirements that ensure the protection of sensitive data. The Federal Information Processing Standards (FIPS) outlines the rules for cryptographic modules employed as mechanism that implement security. A cryptographic module is a combination of hardware and/or software that handles cryptographic functions such as encryption, decryption, digital signatures, authentication techniques and random number generation.

The API gateway pattern has been used as a part of modern software systems for years. A different concept, service mesh, has also emerged over the last couple of years. They share some similarities in their feature set, and service meshes soon started to introduce their own API gateway implementations. In this post, we’ll discuss the Istio ingress gateway, from an API gateway perspective. We’ll examine its feature set compared to typical API gateway features.

Recently, we blogged about certificate management on Kubernetes. Today, we’ll be returning to that topic, but we’ll be focusing on the differences an Istio service mesh makes. The primary difference is the method of solving the ACME HTTP-01 challenge. Solving this challenge involves routing an HTTP request from the ACME server (the Certificate Authority) to the cert-manager challenge solver pod. Cert management with Istio Ingress support As we saw in our previous blog post, you can route such a challenge request by using a Kubernetes Ingress gateway.

Istio claims that it helps to connect, secure, control and observe services. We’ve blogged a lot about connect, even more about observe, and also had a few articles about secure. But so far, we haven’t really touched control. This post tries to fill that gap, and discusses Istio’s access control model, or more specifically AuthorizationPolicies. Architecture Istio Authorization can be used to enforce access control rules between workloads. It basically answers the question: who can access what, under which specific conditions?

Companies frequently use proxies to act as a link between an internal network and the Internet. This is often frustrating for employees, even non-IT ones, when they can’t access a specific site from the company network. For engineers it’s even more obnoxious, since they have to configure all kinds of compute infrastructure to connect to external networks via these proxies. It’s debatable if this is the best way to harden corporate network security, but it’s still the most widely spread method to restrict outgoing traffic.

We recently wrote a very detailed blog post about Kubernetes Ingress. It discusses the various ways of how to route traffic from external sources towards internal services deployed to a Kubernetes cluster. It mostly talks about basic ingress options in Kubernetes, but briefly mentions Istio as a different approach. In this post we examine Istio’s gateway functionality more thoroughly. We discuss the ingress gateway itself that acts as the common entry point for external traffic in the cluster, we take an in depth look into the configuration model, and we finish by talking about the advantages of using Backyards (now Cisco Service Mesh Manager), Banzai Cloud’s production ready Istio distribution.

Backyards (now Cisco Service Mesh Manager) is Banzai Cloud’s widely popular production ready Istio distribution, which helps to install, upgrade, secure, operate, and observe an Istio service mesh. In this blog post, we will discuss the high-level architecture overview of Backyards, three different ways to start using Backyards. Introduction If you’re not familiar with Backyards (now Cisco Service Mesh Manager), and want to know why we decided to build this product, we suggest reading the blog post about the first major release.

Today we’ve launched the 1.3 release of Backyards (now Cisco Service Mesh Manager), Banzai Cloud’s production ready Istio distribution. Along with some performance improvements and bug fixes, the 1.3 release is centered around three main topics: a brand new gateway management feature, a new declarative installation and configuration method, and support for Istio 1.6. If you’re not familiar with Backyards (now Cisco Service Mesh Manager), and want to know why we decided to build this product, we suggest reading the blog post about the first major release.

Istio 1.6 is around the corner and it continues where 1.5 left off: it simplifies the architecture and improves the operational experience. In this post we’ll review what’s new in Istio 1.6 and dig deep on the important changes. The Backyards 1.3 release is already based on Istio 1.6. Istio 1.6 changes We’ll group the changes to three categories: High impact changes (mostly backward incompatible removals) Under the radar changes (smaller changes, but we believe they are interesting/useful) Other changes (honorable mentions) Then, we will detail the first two groups of changes.

Network perimeter security is a focal point of any network admin. When it comes to network perimeter control, our first thought is always inbound security (ingress). However, securing what can leave the network (egress) and where is equally important. In this post, we’re not going to go into the theoretical details of discussing why, exactly, controlling egress traffic is so important or where possible exploitations points are, because there are quite a few posts already.

One of the Istio service mesh’s most popular and robust features is its advanced observability. Because all service-to-service communication is routed through Envoy proxies, and Istio’s control plane is able to gather logs and metrics from these proxies, the service mesh can provide us with deep insights about the state of the network and the behavior of services. This provides operators with unique ways of troubleshooting, managing, and optimizing their services, without imposing any additional burdens on application developers.

When something goes wrong in your mesh, the first thing you’ll probably notice is an alert about your services: error rate or latency is increasing. But it’s only a symptom and the real root cause can be a whole bunch of different things, like underlying Kubernetes problems, application bugs or node failures. This blog post shows you how to track such an issue and find the root cause: in this example, a misconfiguration in a Kubernetes cluster.

Today we are happy to announce the 1.2 release of Backyards (now Cisco Service Mesh Manager), Banzai Cloud’s automated and operationalized service mesh product built on Istio. This is an announcement post describing the new features of Backyards 1.2. If you’re not familiar with Backyards (now Cisco Service Mesh Manager) yet, and want to know why we decided to build this product, we suggest reading the blog post about the first major release.

As the recent CNCF survey suggests (page 7), Istio is one of the most popular service mesh technologies on the market today. The biggest obstacle in Istio’s production adoption so far has probably been that the complexity and domain knowledge required to operate a mesh was too high. The Istio community has realized this and has taken multiple steps to improve the usability and reduce the complexity of Istio. In Istio 1.

Istio, and in general the service mesh has changed the way of service to service communication (from dumb pipes and smart endpoints to sidecar-to-sidecar). While this has lots of benefits, it can increase the complexity of troubleshooting microservice to microservice communication. One of the typical places developers are checking when comes to troubleshoot is the Envoy sidecar proxy container’s access logs, both on the source and destination side. With the increased number of microservices deployed to distributed environments, this process can be tedious, and it’s very inconvenient trying to pair source and destination access log pairs.

Almost every blog post or lecture explaining how Istio service meshes route traffic takes the time to go over how sidecar containers capture outgoing traffic - how that traffic is routed to another service with another sidecar. However, in the real world, a large amount of network traffic passes through the boundaries of the service mesh itself. That traffic might be from a public facing app that receives traffic from the internet, an internal service that needs to connect to a legacy application running outside the mesh, or a workload that consumes an external, third party API.

In today’s blogpost we’re going to be discussing ingress and egress gateways. First, we’ll cover the basics, then we’ll go into detail and explore how they work through a series of practical examples. Ingress and egress gateways are load balancers that operate at the edges of any network receiving incoming or outgoing HTTP/TCP connections. Ingress gateways make it possible to define an entry points into an Istio mesh for all incoming traffic to flow through.

Update: Looks like the solution described below isn’t going to land in Kubernetes after all. The developers went back to the drawing board and they will try to come up with a solution that’s best for everyone. The sidecar concept in Kubernetes is getting more and more popular, and for a good reason. In the container world, it’s a common principle that a container should address a single concern only, but do it well.

One of the challenges we repeatedly faced when using microservices-based solutions was how best to properly secure communication between participating services. One option was to manage security at the application layer, which meant implementing specific authentication mechanisms in the application code itself. This approach, however, would quickly become burdensome, eating up time for developers, who should be concentrating on implementing actual business logic. Wouldn’t it be awesome, we thought, if developers never had to worry about implementing authentication mechanisms in their application code, and, instead, there was a magical solution that would provide secure communication between their services?

Today we are happy to announce the 1.1 release of Backyards (now Cisco Service Mesh Manager), Banzai Cloud’s automated and operationalized service mesh product built on Istio. This is an announcement post describing the new features of Backyards (now Cisco Service Mesh Manager). If you’re not familiar with Backyards (now Cisco Service Mesh Manager) yet, and want to know why we decided to build this product, we suggest reading the blog post about the first major release.

In Kubernetes clusters, the number of Operators and their managed CRDs is constantly increasing. As the complexity of these systems grows, so does the demand for competent user interfaces and flexible APIs. At Banzai Cloud we write lots of operators (e.g. Vault, Istio, Logging, Kafka, HPA, etc) and we believe that whatever system you’re working with, whether it’s a service mesh, a distributed logging system or a centralized message broker operated through CRDs, you will eventually find yourself in need of enhanced observability and more flexible management capabilities.

Two months ago we announced the release of Backyards (now Cisco Service Mesh Manager), Banzai Cloud’s multi- and hybrid-cloud enabled service mesh built on top of our Istio operator. One of Backyards’ hallmarks is its ability to simplify building a production-ready Istio deployment down to a single command: backyards install -a - complete with enterprise grade security, monitoring, tracing, logs, audit, and features like canary releases, traffic management, circuit breaking and lots more, either through a convenient UI, CLI or a GraphQL API.

A while ago we published some benchmarks and sizing about our experience of running Apache Kafka over a service mesh with Koperator and Istio operator, orchestrated by our automated and operationalized service mesh, Backyards (now Cisco Service Mesh Manager). The reasons for such a setup were many, and there are more details in the Running Apache Kafka over Istio - benchmark post, but let me recap some of our initial reasons, and how we evolved from there.

When someone first hears about Istio’s sidecar concept, the first two questions are usually about its effect on resource consumption and request latency or throughput. In this post we’ll discuss the first of those questions: resource consumption of Istio proxies and how it can be reduced with the new Sidecar custom resource. Previously we had a Kafka on Istio benchmark post that was about throughput, and we’ll take a deeper look at the most important HTTP request metrics in a forthcoming article (stay tuned).

Check out Backyards in action on your own clusters! Register for a free version Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Istio has been rightfully praised for ushering in free observability and secure service to service communication.

One of the exciting new features of Istio 1.4 is automatic mutual TLS support, which brings some long awaited convenience to Istio users configuring mTLS for their applications. In this post, we’ll be introducing the concept of Istio’s auto mTLS feature and demonstrating how it works using a demo application. Today, we’ll be using our open-source Banzai Cloud Istio Operator and our multi and hybrid-cloud enabled service mesh platform, Backyards (now Cisco Service Mesh Manager), to install Istio 1.

There is a newer version of Backyards available, read more here: Announcing Backyards 1.1 Today we are happy to announce the first major release of Backyards (now Cisco Service Mesh Manager), Banzai Cloud’s automated service mesh product. Before jumping into the details, we need to answer an obvious question: Why another service mesh product? The service mesh became one of the most hyped things lately, and it may seem that a new product is released every other week.

Check out Backyards in action on your own clusters! Register for a free version Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Service interruptions caused by outages can have severe business consequences, so it’s important that we build, run and test resilient systems.

Check out Backyards in action on your own clusters! Register for a free version Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. In production systems mostly when working in microservices architecture, it is soon realized that monitoring each service individually is often not enough to troubleshoot complex issues.

Check out Backyards in action on your own clusters! Register for a free version Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Istio has been rightfully praised for ushering in free observability and secure service to service communication.

Check out Backyards in action on your own clusters! Register for a free version Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. At Banzai Cloud we work with Istio quite a bit and run a lot of Istio-based service meshes for our customers.

Check out Backyards in action on your own clusters! Register for a free version Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Many organizations are embracing a more decentralized decision making process, allowing them to better build cloud native applications, and to deliver them more rapidly and safely.

UPDATE: For a newer Istio control plane upgrade method using the canary upgrade flow see the Safe and sound canary upgrade for your Istio control plane post. Check out Backyards in action on your own clusters! Register for a free version Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about.

Check out Backyards in action on your own clusters! Register for a free version Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. A few weeks ago we announced Backyards, Banzai Cloud’s automated service mesh built on top of our Istio operator, which greatly simplifies the complex management of service meshes across multi and hybrid-cloud environments.

Check out Backyards in action on your own clusters! Register for a free version Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. In the last few months we wrote a lot of different blog posts about the Istio service mesh.