TY - GEN
T1 - An architecture for fast processing of large unstructured data sets
AU - Franklin, Mark
AU - Chamberlain, Roger
AU - Henrichs, Michael
AU - Shands, Berkley
AU - White, Jason
PY - 2004
Y1 - 2004
N2 - This paper presents a general system architecture tailored to performing searching, filtering, compression, encryption, and other operations on unstructured data streaming from a disk system. The system achieves high performance on such applications by providing for parallelism, hardware-application specialization and reconfiguration, and hardware placement near the disk systems. A limited prototype of a single compute node has been implemented and is described. The prototype is tailored to applications involving complex searching and its performance is compared to a pure software implementation having the same search capabilities. Performance is considered in terms of data set size, query string hit rate and query complexity. Performance results as a function of these parameters are presented and the results indicate that, for data set sizes above 1.4 MB, the prototype compute node is between one and two orders of magnitude faster than a pure software implementation. At high data set sizes, on an individual node, speedups of about 200 and a sustained throughput of 300 MB/sec have been achieved.
AB - This paper presents a general system architecture tailored to performing searching, filtering, compression, encryption, and other operations on unstructured data streaming from a disk system. The system achieves high performance on such applications by providing for parallelism, hardware-application specialization and reconfiguration, and hardware placement near the disk systems. A limited prototype of a single compute node has been implemented and is described. The prototype is tailored to applications involving complex searching and its performance is compared to a pure software implementation having the same search capabilities. Performance is considered in terms of data set size, query string hit rate and query complexity. Performance results as a function of these parameters are presented and the results indicate that, for data set sizes above 1.4 MB, the prototype compute node is between one and two orders of magnitude faster than a pure software implementation. At high data set sizes, on an individual node, speedups of about 200 and a sustained throughput of 300 MB/sec have been achieved.
UR - https://www.scopus.com/pages/publications/17644393218
U2 - 10.1109/ICCD.2004.1347934
DO - 10.1109/ICCD.2004.1347934
M3 - Conference contribution
AN - SCOPUS:17644393218
SN - 0769522319
T3 - Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors
SP - 280
EP - 287
BT - Proceedings - IEEE International Conference on Computer Design
T2 - Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors, ICCD 2004
Y2 - 11 October 2004 through 13 October 2004
ER -