Infrastructure Architecture for SAP HANA and S/4: one customer example

By suggestion of some friends, I've broken the My previous blog post on SAP HANA TDI KPIs in two parts, to give the proper highlight to the architecture considerations I included in the second part of the original version of this post. This is part 2.


          Part 2: One customer example to illustrate

A concrete example: I talked with a customer that was evaluating the migration all his SAP landscape to HANA (not decided yet, just calculating the implications). He had about 45 productive Netweaver based systems, in a total (if I remember well) of about 180 SIDs. Within these, there are 3 productive systems where there are massive data loads, and so write performance is critical. But for all the others, there aren't that many data loads, and the most critical aspect is read performance.

His biggest challenge was to achieve a positive ROI, and the initial architecture proposals we got from some players, fully based on appliances with HANA System Replication for all productive systems, would simply not fit the business case. So he wanted to find ways to build a more economic architecture, while ensuring appropriate service levels to his business.

My suggestion was to define as a standard, a virtualized architecture on top of a stretched cluster replicated across 2 datacenters at a "metro distance". For you more knowledgeable in infrastructure, the replication between the sites was already being done using EMC VPLEX Metro.
This architecture would most likely serve for 177 of his 180 systems.

This architecture drove major cost savings, and depending on how much saving the customer wanted to draw out of this scenario as well as the distance between the 2 replicated datacenters, it might happen that some of the SAP HANA VMs might not fulfill all the TDI KPIs. And that is no big deal, as this kind of compromise is already happening today in the Netweaver world, and is good enough for organizations, better when put in perspective with the major savings achieved.

Then as the other 3 were an exception, we came to a discussion in regards to what would be the acceptable balance between performance and data resiliency.

If you want zero Recovery Point Objective (RPO), meaning no data loss, there is no other way that to implement some sort of synchronous replication. Being the replication done at the storage level or through HANA System Replication, you'll always suffer the impact of the network latency.

If you cannot compromise performance because the load on this system will be massive, no way you'll be able to replicate synchronously to a datacenter at a "metro distance" as the network latency will always add up degrading SAP HANA write performance.

Then, you need to evaluate your specific business scenario, and define which is the best compromise.


          2 Step SLA as a compromise solution for the "performance and availability" paradigm

Through the discussion I suggested him then to define a "2 level SLA":

• One SLA for single component failure within a datacenter;
• And another SLA for a full site loss.

When you build resiliency into your architecture, what you're doing is putting in place a contingency for a business risk.

All risks are a factor of probability and impact.

So, the probability of a single component in the datacenter infrastructure failing is very high, but if you design properly the systems, the impact can be minimal.

Also, the probability of loosing a full datacenter is very low, although the impact would be massive.

So, maybe the RTO and RPO you would be willing to accept in these two scenarios would be different, as if you lose a full site, maybe this is a major disaster, and it might even imply that your customers would be out of business as well, so you need to evaluate how much are you willing to spend to put a contingency in place, or whether is acceptable to reduce your SLAs and complement them with an insurance policy for unexpected losses.

In the end, all this discussion goes back to ensuring that you fulfill business needs, while at the same time you ensure that the TCO of your solution doesn't become bigger than the business benefits of it.

This then opened the following possibility as a more costly one, but one that would balanced the extreme availability and performance requirements of these systems. Doing this for the few systems that "really required such extreme characteristics" enabled a major cost saving on all the other systems included in the above defined standard (the virtual Datacenter).

To protect from a single component failure, it could be implemented SAP HANA synchronous replication with "in-memory pre-load" option, ensuring zero RPO and near zero RTO (less that 1 minute for sure). This would be done across 2 servers in the same datacenter, to ensure minimal performance impact.

For full datacenter loss, then one of the systems would be replicating asynchronously at the storage level to the secondary datacenter. The calculated RPO would be less than 1 minute and the calculated RTO would be less than 15 minutes, ensuring an acceptable SLA, with minimal performance impact, and also reducing the TCO by allowing the utilization of the system on the replicated datacenter to be used for "pre-production testing" in normal operations.


          Conclusion

Not all business systems are alike, and not all have the same business requirements. Being them running on SAP HANA or not!!!

This implies that the technical requirements will not be the same to all these systems either.

By defining a standard for the majority of the SAP HANA and Netweaver systems based on Virtualization and Mirrored Datacenters with VPLEX, this organization would achieve a very flexible architecture, easily manageable, with a very high utilization of assets, and so with a very high service level at a very affordable cost.

Even if some of the systems would not fulfill all the TDI KPIs, it is not a big deal, as all this KPI discussion is one of "performance" and not of functionality, or even support as stated in SAP's documents referenced in this blog.

Then exceptions were evaluated for what they are: EXCEPTIONS!

And specific solutions were design according to the business requirements.

So, again, SAP HANA is becoming a normal application in the datacenter! Make sure to keep up to date on the current rules and conditions to ensure the most suitable architecture to your organization, not only from a performance perspective, but also from a data resiliency, availability, CAPEX and most of all OPEX, as its OPEX you'll be living with for the rest of your solution life.

Being this just one example, I hope this helps others going through the same discussion.

SAP HANA TDI KPI: mandatory or indicative? And what about the network implications on Synchronous replication?

By suggestion of some friends, I've broken this blog post in two to give the proper highlight to the architecture considerations I included in the second part of the original version of this post. That seccond part is now published at: http://sapinfrastructureintegration.blogspot.com/2015/10/exceptions-are-exceptions-one-example.html


          Part 1: The current facts

I've been receiving a lot this question: when I have SAP HANA either with storage or HANA System Replication in a synchronous mode, depending on the network latency, it will not meet the TDI KPIs. Will SAP accept this? What would be the performance impact at an application level?

The question came as this customer wanted all the performance he could get, but also the maximum data resiliency technology could provide today.

In a nutshell, there are certain situations that the "Laws of Physics" do not allow you to have all you want, and you need to chose: either the maximum performance, or the maximum data resiliency.

So, going through this reasoning I first collected the currently available information from SAP in this regards, just to reach the conclusion that "YES it is acceptable to fail KPIs" as SAP says "its the customer's decision to define whether the performance penalty is acceptable to his specific business scenario". Then, I entered a discussion with him on finding the right balance between technical and business requirements for his architecture blueprint.

Where did I get the information that it is acceptable to fail KPIs?

Let me share here what I wrote some weeks ago to a customer with a question in regards to what latency is accepted when doing storage replication, as a competitor was telling him that if he had storage replication and did not fulfill all the TDI KPIs, he could not run any production workload on that infrastructure, which is FALSE!

Then, I'll also share the reasoning that followed after getting this question cleared up, by providing a concrete customer example I hope helps you build your own reasoning in case you're going through a similar discussion in your organization as well.

          Network latency impact on SAP HANA performance when replicating synchronously

Indeed the network latency will be a critical factor on SAP HANA performance. But it is both for storage replication in the same way it is in the case the customer decides to implement SAP HANA System Replication in synchronous mode.

My advice here is to carefully evaluate what is the latency between the two sites being synchronously replicated. Ideal is in fact for latency to be below 1ms. More than that can really start to impact write performance, so if the customer application will have massive data loads, the data load time can become longer because of this. There will be tough, absolutely no impact on the read performance as it is done in the memory of the server. So depending on your application specific workload profile, you may feel a lot this increased latency or nothing at all.

In the end, all the details are on SAP’s Whitepapers.
There are two in particular relevant to this matter: The SAP HANA Network Requirements whitepaper: http://scn.sap.com/docs/DOC-63221
In page 28 it says:

• There is no straightforward recommendation regarding the bandwidth and the latency a system replication network must provide. A rough estimation of the required performance is given in the How-to Guide Network Required for SAP HANA System Replication, which is referred to from the SAP Note 1999880 - FAQ: SAP HANA system replication
• Latency: The redo log shipping time for 4 KB log buffers must be less than a millisecond or in a low single-digit millisecond range – depending on the application requirements (relevant for synchronous replication only).

This applies to storage replication as well. So, let’s say that if you cannot stay below 1ms, that 2 or 3 ms may be acceptable. But it will depend on your specific business scenario.
Transactional scenarios (like SAP ERP on HANA) are more sensitive to latency. Analytical scenarios are more sensitive to throughput.

Looking at the note mentioned above it sends you in a sequence of documents being the final one: Network Recommendations for SAP HANA System Replication at http://scn.sap.com/docs/DOC-56044
Here, the same as above is mentioned:

• All changes to data are captured in the redo log. The SAP HANA database asynchronously persists the redo log with I/O orders of 4 KB to 1 MB size into log segment files in the log volume (i. e. on disk). A transaction writing a commit into the redo log waits until the buffer containing the commit has been written to the log volume. This wait time for 4 KB log buffers should be less than a millisecond or in a low single-digit millisecond range.

          Even without replication, it is your choice whether is acceptable to fail KPIs

Let me add as a final note the transcript of the response to question 7 on page 7 of the SAP HANA TDI FAQ published at SCN: SAP HANA TDI - FAQ | SCN

There is written:

Q: Which cases where one or more KPIs are not fulfilled does SAP consider as uncritical or acceptable?

• This is always up to the customer to decide if falling below the KPIs is acceptable for his/her daily operation of the SAP HANA system. The customer must decide whether the performance of his/her SAP HANA system is sufficient for his/her needs.
• The following questions and examples might help for making that decision:
  • Is the given SAP HANA system a non-productive system?
    • The KPIs apply for production systems only. In general, for non-productive systems weaker performance is acceptable
  • Which SAP HANA scenarios are run on the given system?
    • OLAP scenarios only (e.g. SAP BW-on-HANA):
      • The performance of queries is usually not affected if all required tables have been loaded into memory
      • Latency times of the log volume and throughput rates for writing the data volume are mainly relevant when loading data from source systems
      • Throughput rates for reading from the data or the log volume affect the overall system restart time (e.g. after applying an SAP HANA revision update)
    • OLTP scenarios only (e.g. SAP Business-Suite-on-HANA):
      • Latency times of the log volume affect the duration of every transaction that changes one or more tables in the database
      • Throughput rates for reading from the data or the log volume affect the overall system restart time (e.g. after applying an SAP HANA revision update)
    • See the Storage Requirements whitepaper for details: SAP HANA TDI - Storage Requirements | SCN

So, as you can see, SAP HANA is really becoming a normal application in the datacenter, and more choice is given to customers every single day. Today, even without replication, it is each customer's choice whether failing certain KPIs is acceptable.

And this is relevant for example when you think on very small SAP HANA systems that are not very critical, that you see more and more as customers make SAP HANA mainstream and migrate every single system to HANA. There may be a couple of systems that are very big and very critical, but there are a ton of others where you could compromise a bit of performance to have a better TCO.