Samsung’s Software Centre in Korea and Samsung Research America (SRA) in the Silicon Valley have developed a new sorting engine called DeepSort. Today, Samsung Electronics announced that its latest sorting engine, DeepSort, has topped Sort Benchmark’s Minute Sort test with record-breaking results. Minute Sort is an industry benchmark which tests out the amount of data an algorithm can sort in 60 seconds; Samsung’s DeepSort analysed and reorganised 3.7 terabytes of data in a minute, considering the previous record of 1.5 terabytes, this is a monumental achievement for the Korean giant.
DeepSort was tested in a networked environment composed of 384 nodes. Each node was packed with two 2.1GHz Intel Xeon hexa-core processors, 64GB RAM, eight 3TB 7200RPM hard drives with ext4 file system, CentOS 6.4, and one 10Gbps Ethernet port. By now, your jaw has probably been dropped to the floor by looking at those specifications, and you want one of those nodes in your house for all your complex processing needs. Well, it turns out, these are actually moderate specifications when compared to other main competitors in the competition. Samsung chose to take this route, so that Sort Benchmark could more accurately evaluate its software’s pure performance.
With DeepSort, Samsung is targeting high efficiency at a large scale. To achieve this goal, Samsung has enabled simultaneous processing and multithreading at all stages and components of the program; a fluent data flow that shares the limited memory space and minimises data move-moment. This allows the Korean giant to evenly distribute the load among all servers; minimising buffering time in the servers’ hard drives by maximising memory cache usage.
“Billions of people across the world continue to generate an incredible amount of data,” said Juhan Lee, Vice President at the Samsung Electronics Software Center in Seoul. “As a result, data processing capabilities have become fundamental in developing next-generation services for consumers, such as those related to the Internet of Things and digital health. DeepSort and other innovative solutions currently in the pipeline will form the foundation upon which a new generation of such services may reach their full potential.”
Samsung is expected to implement DeepSort into a number of its mobile applications and services that incorporate search functions. This, in return, should allow applications to fetch records faster than before, hence, reducing the wait time between search queries; ultimately improving the user experience.
PRESS RELEASE
Seoul, KOREA – November 26, 2014 – Samsung Electronics Co. Ltd., today announced its record-breaking sorting engine, DeepSort, finished No. 1 in this year’s Sort Benchmark competition, often known as the “World Cup” of data sorting. The competition, which pitted Samsung against other major technology companies and research institutions, saw Samsung’s DeepSort analyze and reorganize 3.7TB of data in 60 seconds, more than doubling the previous record of 1.5TB in the MinuteSort benchmark, an industry benchmark on the amount of data a system can sort through in a minute.
“Billions of people across the world continue to generate an incredible amount of data,” said Juhan Lee, Vice President at the Samsung Electronics Software Center in Seoul. “As a result, data processing capabilities have become fundamental in developing next-generation services for consumers, such as those related to the Internet of Things and digital health. DeepSort and other innovative solutions currently in the pipeline will form the foundation upon which a new generation of such services may reach their full potential.”
DeepSort, technically referred to as a distributed general sorting engine, was tested in a networked environment composed of 384 servers. In order to more accurately gauge the software’s sheer performance, hardware and software of moderate specifications were used throughout the test. For instance, the number of servers, the 64GB memory capacity and 2.1GHz processing power of each server were significantly lower relative to those of the main competitors in this year’s Sort Benchmark contest, while hard disk drives (HDD) were used in place of significantly faster solid state drives (SSD).
The record-breaking performance of DeepSort was attributed to fine-tuning a range of key elements in accelerating data processing speed, which occur throughout the overall sorting process. They include, among others, preventing additional wait time for servers with relatively higher data loads by first evening out the sorting load among all servers; enabling simultaneous processing, communication and storage of data by concurrently implementing thousands of threads; and minimizing buffering time in the servers’ HDDs by maximizing memory cache usage.
Developed jointly by Samsung’s Software Center in Korea and Samsung Research America (SRA) in the Silicon Valley, DeepSort is expected to be implemented into a number of Samsung services and mobile apps that incorporate search functions.
DeepSort’s achievement comes in the midst of Samsung’s ongoing push into software innovation, an initiative for which the company has dedicated significant financial investments and more than 40,000 software developers with diverse backgrounds and capabilities.
