STH turns 7 and announcing the biggest new feature in the site’s history

In 2009 STH started as a personal blog around the idea of building a large scale home storage server. Today, STH is one of the leading enterprise technology sites on the Internet. Every anniversary I get to write this post, my favorite of what I hope will be a continuing series. In the last year we had a number of accomplishments that I wanted to briefly highlight. Looking ahead, as of today, STH is officially announcing a new class-leading capability that I know many of you will be extremely excited for. This launch is a driving factor that led me to leave a global management consulting role at PricewaterhouseCoopers and focus on building STH full time.

Highlights for STH in Year 6

STH saw a major expansion in its sixth year of existence. Here are a few of the highlights:

• The site itself grew by approximately 75% in terms of traffic. Metrics such as forum postings per day have more than doubled in the last year.
• Held our first in-person #STHmeetup event April 2016 which had double the attendance we were hoping to get in our stretch goal.

  

• I left PwC to focus on STH and DemoEval full time.
• Expanded participation on the site. We have received a lot of “anonymous” help beyond the articles published on the site. For those that wish to stay anonymous, thank you.
• Primary hosting moved out of Fiberhub and into a different facility and changed architectures. We use ZFS and Ceph storage along with KVM virtualization.
• We expanded relationships with a host of new vendors and have, or will be reviewing products in the near future. See our list of companies we work with here.
• Successfully built a new shared demo lab. You can see the now slightly dated statistics in our May 2016 update.
• We had a number of world first third party benchmarks and reviews (e.g. the vast majority of the Intel Xeon D-1500 and E5-2600 V4 lines.)

That is now part of STH’s history and a successful year by just about any metric. It is time now to focus on the future.

Areas we are expanding to in 2016-2017

I have a few areas I wanted to update the STH community on. Admittedly, we do change focus areas as new developments happen, but here are a few we are working on:

• Storage – we have a new storage test methodology and have the high-speed networking and test nodes available to really scale this up. Specifically, looking at scale out architectures.
• Networking and Security – this has been an area I have been reluctant to focus on due to it feeling like too much of the “family business” in past years. That feeling has largely subsided.  We are looking for someone to take over switch/ router/ appliance reviews on a more regular basis. In the meantime, we have been expanding our coverage in terms of reviewing switches and firewalls. The security side is going to be a heavy investment focus area so we are expanding coverage.
• Traditional Compute – This is a core area for STH but we are building a next-generation of Linux-Bench to better test server workloads.
• Machine learning/ Deep learning – We have been starting to work in this area. I feel extremely passionate about this being “the next big application” if it is not already.

Lastly, there is the immediate expansion with DemoEval.

Announcing DemoEval

The benchmarking world is fascinating. I could certainly write a book with just the stories over the past year. While we can profile benchmarks and scaling among hundreds of platforms (and we do) there is always room for the “rogue” optimization of code because someone re-wrote code in assembly in an open source package that biases performance towards one architecture.

Looking at industry standard benchmarks, e.g. Linpack, there is a problem emerging. The world has created so much code at this point and the increased use of automated code generation has lead to a common problem of “that is great, but it is not my workload.” Even more on point, you can run a nginx benchmark but there are so many companies with custom nginx version (or other stack components) out there that the question and answer has become a lot more individualized. Multiple Oracle and SAP implementations I have worked on in the past years have had over $100 million USD in customization for a single company, and just in the Quote to Cash process. Although I was not the one doing the coding, a $100 million implementation job at a Fortune 500 company means there is a lot of customized code.

The industry standard way to fix the “but this is not my workload” problem is twofold. One, a company can work with a vendor or reseller and get a demo machine for evaluation. Either physical or virtual and run their own code. Two, a new benchmark is built that will inevitably return to the “it is not my workload issue.” Today we are expanding options to cover a third option: DemoEval. This is a project we briefly described in our May 2016 lab update. It has been very successful thus far as we showed in September 2015 to May 2016 growth:

We have a full write up in the Announcing DemoEval – The Future of Technology Reviews article on STH and on the DemoEval site. Head to those resources to see more about the platform. For those who simply want a brief synopsis, we are going to allow you to test drive systems directly in our demo lab. We have been working on iterations of the platform for almost a year and it is finally something that is extremely easy to use. If you want to compare the performance of a machine you use today against a new machine we have reviewed, you can sign up and run your own workload on the machines. This can help form business cases to buy the machines (or other hardware) or even to simply try out new platforms and technologies on a regular basis.

There is no waiting on the vendor, having to rack and cable gear, or running gear at home. You can run it in our lab. Since we often publish our reviews of platforms the day that they are announced, we are giving vendors the option of allowing us to make that gear available in our demo program. As we have discussed this capability with vendors I have been clear, I have no interest in becoming a reseller at this point. The plan is for this to remain an independent lab facility.

Check out the new functionality in our STH Announcement or the DemoEval website.