We have heard your feedback: you want to run your own workloads on the hardware we test at STH. Today we are announcing DemoEval, our platform for running your workloads on cutting edge platforms. Simply put, if you have read a review on STH or elsewhere, and thought “I wish I could try that system out,” now you can. We are using our experience with working with Fortune 500 organizations, the review community and Silicon Valley start-ups to bring a platform to a much broader audience.
DemoEval Overview
We are curating a menu of interesting systems for DemoEval subscribers to try. The key here is curating the menu. These will not be poorly equipped systems (e.g. a 10 core CPU with 4GB of RAM and where storage is a single 1TB hard drive.) We are going to bring a menu of the newest technology to our subscribers where they have the opportunity to run their own workloads on a realistic server.
With your monthly subscription, you can reserve one of these systems for a 5-day work week each month (Monday through Friday). Saturday/ Sunday are currently reserved for resetting hardware/ login credentials, wiping disks and related activities. We do plan to offer shorter duration subscriptions at a later date.
The platform already has 10/40Gb Ethernet networking (and FDR Infiniband) available. It has a 1Gbps WAN connection for a very limited number of machines and this will grow as we add more machines to ensure plenty of WAN capacity. We are also working to bring 25, 50 and 100GbE networking into the lab.
As a subscriber you will get an in-data center virtual desktop to connect to the system you are evaluating. These desktops can be accessed from HTML5 browsers or VDI clients. We do have WAN acceleration technology to handle even higher latency connections. There are local ISO mirrors for popular Linux distributions. You will have bare metal access to the system or systems you are testing. We have had folks login and show their bosses testing sessions at the beach, at breakfast over 4G LTE, airports and have tried it at trade shows over 6,500 miles away.
Although you can install a local client, it will work fine within a browser setting. We will pre-load your desktop with links to the relevant management interfaces and any additional collateral required to access a given machine.
How did this start?
The platform originally was developed to solve a single problem for us: safely review very expensive hardware and provide how-to guides using this hardware. Many of the machines we test sell for tens of thousands of dollars. They are too costly for us to ship and have tested by subject matter experts.
In April 2015 we started with a single low power rack for STH use. In September 2015 we quadrupled our capacity to multiple racks. In addition, we started to allow other professional reviewers in the industry access our infrastructure to get their feedback on the setup. We have also expanded to an additional data center to enhance capacity.
You can see a picture of the in-progress updates in our May 2016 data center lab update.
Over the past year we have iterated at least a dozen times to arrive at a platform that others can use easily. We have formally been in beta with a half a dozen users over the past few weeks and the current iteration is ready to go. These have ranged from individuals working at IT consulting firms, corporate IT at large companies, vendors as well as startup stage companies evaluating on-prem v. cloud hosting.
The road to arrive at this platform was not as quick and easy as we thought when we first set out on the project. We now have something that works well. We have been using it daily to test everything from low end switches to multiple blade servers.
DemoEval FAQ
Here is the DemoEval FAQ:
Will every server and component STH reviews be in the DemoEval program?
We will only place a system in the DemoEval program if we get permission from the vendor on a vendor supplied system. We do purchase hardware from time to time that we can host without vendor approval. We are curating configurations which means we are actively adding components to review systems to make them more interesting.
Why do we charge?
We do charge for the service for a number of reasons. The primary reason is that we are often buying hardware to test or augment systems or components we receive. We also have to receive, rack and cable the systems and keep them in a neutral data center. We then have to pull, re-pack and ship systems back once they are no longer needed in the program. To maintain a high-end platform, we have already invested almost $200,000 in the infrastructure and are continuing to invest in the experience. With high-end systems in a data center with a fast WAN connection there is also a great opportunity for abuse that we need to curb.
The DemoEval service is meant to be paid for by a workplace, not individuals.
Is the only access option paid?
No. We are still using this infrastructure to review new gear and produce guides for STH. Anyone who would like to use the infrastructure in exchange for building guides should send me a proposal (patrick at this domain) with the following information:
- Proposed article title
- Proposed outline for the topic
- DemoEval hardware required
- Duration system(s) are needed for
We will review each request and approve based on the need we have and availability of hardware.
Can I run production workloads on the service?
This service is intended to help you evaluate enterprise hardware. While we are taking some basic security measures, you should assume these are public facing servers. As a result, we ask that you use sanitized data and refrain from running production workloads. We are also going to wipe each system on weekends. If you have production data on the server it will be lost forever. There are many hosting providers who can help setup production ready systems based on what you try in DemoEval.
So are you a reseller now?
Absolutely not. We have been very explicit with vendors that this is a third party independent facility. We have no plans to become a reseller outfit. We will not exclude resellers from subscribing, but we will not sell you the machines. If you do need a referral, we are happy to do so as that lets vendors gauge effectiveness of the platform. Remaining an independent facility allows us to cover gear from any vendor and frees us from having quota pressures.
How can one get started?
If you are interested in DemoEval check out the DemoEval website for more information.
