At ISC 2022, STH visited with YellowDog. The company provides a tool that allows one to easily manage cloud instances. There are a number of cloud management platforms, but YellowDog has some interesting features, so we figured we would do a quick write-up.
YellowDog Cloud Instance Management at ISC 2022
The premise behind YellowDog is that it allows one to manage instances across a number of users, clouds, and on-prem servers.
One can set up parameters such as pipelines that are automatically triggered when a new file is loaded in an object store. When the file is loaded, a pipeline can be triggered to spin up instances and work on the new file, and then shut down when there is no more work.
A cool feature is that one can put a set of constraints, memory, vCPUs, storage, and etc, on where the workload will run. It can then source instances across a number of clouds. That can also include spot instance pricing. Then, the tool builds a list of the closest fit instances and can rank them by fit and cost. That way if an instance (for example spot) is the cheapest, but is not available, it can use the list to find the next-best alternative across different providers.
As jobs are run, the tool can report usage information to monitor how cloud spend is happening.
The tool also has the ability to manage quotas or allowances for different clouds, users, groups and so forth.
Aside from the cost and workload management features, the tool gives visibility across cloud spend, an increasingly important rate to monitor.
Final Words
While to many this may not be as exciting as the latest CPU, GPU, or accelerator, having a tool like YellowDog allows one to manage costs. Perhaps the bigger implication will happen in the coming months. As we move to an era where there are massive differences in accelerator technologies, tools like YellowDog that can help find and prioritize instances with accelerators will become more important. Already there is a difference in vCPUs when it comes to accelerators as we showed in Stop Leaving Performance on the Table with AWS EC2 M6i Instances. As future platforms have more accelerated than past platforms, it may end up being that a 2 vCPU instance with an accelerator outpaces a 16 vCPU instance without the accelerator. That will make this type of technology more important in the coming years.
Does yellow dog have anything to do with the Linux for PowerPC yellow dog?
Does it let you import your existing fleet and start managing them, or do you need to launch new everything-s via YellowDog?
Yet another management tool.. but.. whats new? Pffff..