# RecSys ## expand\_more ## Architectural Support for Deep Recommendation Systems(RecSys) ![RecSys Figure 1](/sites/g/files/omnuum11281/files/styles/hwp_1_1__720x720_scale/public/vlsiarch/files/recsys_figure_1.png?itok=x_0fwNnk) Recommendation systems form the backbone of popular internet services like entertainment streaming, e-commerce and social media (e.g., Netflix, Amazon, Facebook). These deep learning-based systems not only present unique compute challenges compared to well-studied DNNs but also introduce significant infrastructure demands for at-scale deployment. In order to make recommendation more efficient, solutions will have to integrate insights across the entire execution stack (as shown below). At the use-case level, our group looks at optimizing both training and inference cycles with datacenter-scale (e.g., workload scheduling) as well as mobile-centric (e.g., privacy preservation) solutions. Framing the problem in these specific contexts allows us to propose algorithmic adjustments (e.g., model compression and partitioning) that are most appropriate for each use case. Last but not least, we quantify the implications of current heterogeneous hardware designs on recommendation workloads and use these insights to propose future architectures specialized for recommendation. ## Select Publications Download 7 citations download- [BibTeX](/bibcite/export?pager_style=standard_pager&number_of_items=20&sort_field=bibcite_year--desc&taxonomy_filters%5Bfield_hwp_c_peoplepublications%5D&taxonomy_filters%5Bfield_hwp_c_project123456%5D%5B0%5D%5Btarget_id%5D=172593&&&format=bibtex) - [EndNote X3 XML](/bibcite/export?pager_style=standard_pager&number_of_items=20&sort_field=bibcite_year--desc&taxonomy_filters%5Bfield_hwp_c_peoplepublications%5D&taxonomy_filters%5Bfield_hwp_c_project123456%5D%5B0%5D%5Btarget_id%5D=172593&&&format=endnote8) - [EndNote 7 XML](/bibcite/export?pager_style=standard_pager&number_of_items=20&sort_field=bibcite_year--desc&taxonomy_filters%5Bfield_hwp_c_peoplepublications%5D&taxonomy_filters%5Bfield_hwp_c_project123456%5D%5B0%5D%5Btarget_id%5D=172593&&&format=endnote7) - [Endnote tagged](/bibcite/export?pager_style=standard_pager&number_of_items=20&sort_field=bibcite_year--desc&taxonomy_filters%5Bfield_hwp_c_peoplepublications%5D&taxonomy_filters%5Bfield_hwp_c_project123456%5D%5B0%5D%5Btarget_id%5D=172593&&&format=tagged) - [Marc](/bibcite/export?pager_style=standard_pager&number_of_items=20&sort_field=bibcite_year--desc&taxonomy_filters%5Bfield_hwp_c_peoplepublications%5D&taxonomy_filters%5Bfield_hwp_c_project123456%5D%5B0%5D%5Btarget_id%5D=172593&&&format=marc) - [PubMedId](/bibcite/export?pager_style=standard_pager&number_of_items=20&sort_field=bibcite_year--desc&taxonomy_filters%5Bfield_hwp_c_peoplepublications%5D&taxonomy_filters%5Bfield_hwp_c_project123456%5D%5B0%5D%5Btarget_id%5D=172593&&&format=pubmed_id) - [RIS](/bibcite/export?pager_style=standard_pager&number_of_items=20&sort_field=bibcite_year--desc&taxonomy_filters%5Bfield_hwp_c_peoplepublications%5D&taxonomy_filters%5Bfield_hwp_c_project123456%5D%5B0%5D%5Btarget_id%5D=172593&&&format=ris) ### 2021 Udit Gupta, Samuel Hsia, Jeff Zhang, Mark Wilkening, Javin Pombra, Hsien-Hsin S. Lee, Gu-Yeon Wei, Carole-Jean Wu, and David Brooks. 2021. “[RecPipe: Co-Designing Models and Hardware to Jointly Optimize Recommendation Quality and Performance](/publications/recpipe-co-designing-models-and-hardware-jointly-optimize-recommendation)”. MICRO ’21: MICRO-54: 54th Annual IEEE ACM International Symposium on Microarchitecture, Pp. 870–884 Udit Gupta, Samuel Hsia, Jeff Zhang, Mark Wilkening, Javin Pombra, Hsien-Hsin S. Lee, Gu-Yeon Wei, Carole-Jean Wu, and David Brooks. 2021. “[RecPipe: Co-Designing Models and Hardware to Jointly Optimize Recommendation Quality and Performance](/publications/recpipe-co-designing-models-and-hardware-jointly-optimize-recommendation)”. MICRO ’21: MICRO-54: 54th Annual IEEE ACM International Symposium on Microarchitecture, Pp. 870–884 - add\_circle\_outline do\_not\_disturb\_on Abstract - [ descriptionPublisher's Version](https://doi.org/10.48550/arXiv.2105.08820) - [ picture\_as\_pdfRecPipe: Co-designing Mod...](/sites/g/files/omnuum11281/files/vlsiarch/files/2105.08820.pdf) Deep learning recommendation systems must provide high quality, personalized content under strict tail-latency targets and high system loads. This paper presents RecPipe, a system to jointly optimize recommendation quality and inference performance... - [ descriptionPublisher's Version](https://doi.org/10.48550/arXiv.2105.08820) - [ picture\_as\_pdfRecPipe: Co-designing Mod...](/sites/g/files/omnuum11281/files/vlsiarch/files/2105.08820.pdf) Mark Wilkening, Udit Gupta, Samuel Hsia, Caroline Trippel, Carole-Jean Wu, David Brooks, and Gu-Yeon Wei. 2021. “[RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference](/publications/recssd-near-data-processing-solid-state-drive-based-recommendation-inference)”. ASPLOS 2021: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Pp. 717–729 Mark Wilkening, Udit Gupta, Samuel Hsia, Caroline Trippel, Carole-Jean Wu, David Brooks, and Gu-Yeon Wei. 2021. “[RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference](/publications/recssd-near-data-processing-solid-state-drive-based-recommendation-inference)”. ASPLOS 2021: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Pp. 717–729 - add\_circle\_outline do\_not\_disturb\_on Abstract - [ descriptionPublisher's Version](https://doi.org/10.48550/arXiv.2102.00075) - [ picture\_as\_pdfRecSSD: Near Data Process...](/sites/g/files/omnuum11281/files/vlsiarch/files/2102.00075.pdf) Neural personalized recommendationmodelsareusedacrossawide Samuel Hsia Harvard University Cambridge, Massachusetts, USA David Brooks Harvard University Cambridge, Massachusetts, USA USA. ACM, New York, NY, USA... - [ descriptionPublisher's Version](https://doi.org/10.48550/arXiv.2102.00075) - [ picture\_as\_pdfRecSSD: Near Data Process...](/sites/g/files/omnuum11281/files/vlsiarch/files/2102.00075.pdf) ### 2020 Samuel Hsia, Udit Gupta, Wilkening Mark, Carole Wu, Gu-Yeon Wei, and David Brooks. 2020. “[Cross-Stack Workload Characterization of Deep Recommendation Systems](/publications/cross-stack-workload-characterization-deep-recommendation-systems-0)”. In 2020 IEEE International Symposium on Workload Characterization (IISWC) Samuel Hsia, Udit Gupta, Wilkening Mark, Carole Wu, Gu-Yeon Wei, and David Brooks. 2020. “[Cross-Stack Workload Characterization of Deep Recommendation Systems](/publications/cross-stack-workload-characterization-deep-recommendation-systems-0)”. In 2020 IEEE International Symposium on Workload Characterization (IISWC) - add\_circle\_outline do\_not\_disturb\_on Abstract - [ descriptionPublisher's Version](https://arxiv.org/abs/2010.05037) - [ picture\_as\_pdfCross-Stack Workload Char...](/sites/g/files/omnuum11281/files/vlsiarch/files/2010.05037.pdf) Deep learning based recommendation systems form the backbone of most personalized cloud services. Though the computer architecture community has recently started to take notice of deep recommendation inference, the resulting solutions have taken wildly... - [ descriptionPublisher's Version](https://arxiv.org/abs/2010.05037) - [ picture\_as\_pdfCross-Stack Workload Char...](/sites/g/files/omnuum11281/files/vlsiarch/files/2010.05037.pdf) Udit Gupta, Samuel Hsia, Vikram Saraph, Xiaodong Wang, Brandon Reagen, Gu-Yeon Wei, Hsien-Hsin S. Lee, Carole-Jean Wu, and David Brooks. 2020. “[DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference](/publications/deeprecsys-system-optimizing-end-end-scale-neural-recommendation-inference)”. In . The 47th IEEE/ACM International Symposium on Computer Architecture (ISCA 2020) Udit Gupta, Samuel Hsia, Vikram Saraph, Xiaodong Wang, Brandon Reagen, Gu-Yeon Wei, Hsien-Hsin S. Lee, Carole-Jean Wu, and David Brooks. 2020. “[DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference](/publications/deeprecsys-system-optimizing-end-end-scale-neural-recommendation-inference)”. In . The 47th IEEE/ACM International Symposium on Computer Architecture (ISCA 2020) - add\_circle\_outline do\_not\_disturb\_on Abstract - [ descriptionPublisher's Version](https://conferences.computer.org/isca/pdfs/ISCA2020-4QlDegUf3fKiwUXfV0KdCm/466100a982/466100a982.pdf) - [ picture\_as\_pdfDeepRecSys: A System for ...](/sites/g/files/omnuum11281/files/vlsiarch/files/2001.02772.pdf) Neural personalized recommendation is the corner-stone of a wide collection of cloud services and products, constituting significant compute demand of the cloud infrastructure. Thus, improving the execution efficiency of neural recommendation directly... - [ descriptionPublisher's Version](https://conferences.computer.org/isca/pdfs/ISCA2020-4QlDegUf3fKiwUXfV0KdCm/466100a982/466100a982.pdf) - [ picture\_as\_pdfDeepRecSys: A System for ...](/sites/g/files/omnuum11281/files/vlsiarch/files/2001.02772.pdf) Liu Ke, Udit Gupta, Carole-Jean Wu, Benjamin Cho, Mark Hempstead, Brandon Reagen, Xuan Zhang, David Brooks, Vikas Chandra, Utku Diril, Amin Firoozshahian, Kim Hazelwood, Bill Jia, Hsien-Hsin Lee, Meng Li, Bert Maher, Dheevatsa Mudigere, Maxim Naumov, Martin Schatz, Mikhail Smelyanskiy, and Xiaodong Wang. 2020. “[RecNMP: Accelerating Personalized Recommendation With Near-Memory Processing](/publications/recnmp-accelerating-personalized-recommendation-near-memory-processing)”. In . The 47th IEEE/ACM International Symposium on Computer Architecture (ISCA 2020) Liu Ke, Udit Gupta, Carole-Jean Wu, Benjamin Cho, Mark Hempstead, Brandon Reagen, Xuan Zhang, David Brooks, Vikas Chandra, Utku Diril, Amin Firoozshahian, Kim Hazelwood, Bill Jia, Hsien-Hsin Lee, Meng Li, Bert Maher, Dheevatsa Mudigere, Maxim Naumov, Martin Schatz, Mikhail Smelyanskiy, and Xiaodong Wang. 2020. “[RecNMP: Accelerating Personalized Recommendation With Near-Memory Processing](/publications/recnmp-accelerating-personalized-recommendation-near-memory-processing)”. In . The 47th IEEE/ACM International Symposium on Computer Architecture (ISCA 2020) - add\_circle\_outline do\_not\_disturb\_on Abstract - [ picture\_as\_pdfRecNMP: Accelerating Pers...](/sites/g/files/omnuum11281/files/vlsiarch/files/1912.12953.pdf) Personalized recommendation systems leverage deep learning models and account for the majority of data center AI cycles. Their performance is dominated by memory-bound sparse embedding operations with unique irregular memory access patterns that pose a... - [ picture\_as\_pdfRecNMP: Accelerating Pers...](/sites/g/files/omnuum11281/files/vlsiarch/files/1912.12953.pdf) Udit Gupta, Carole Wu, Xiaodong Wang, Maxim Naumov, Brandon Reagen, David Brooks, Bradford Cottel, Kim Hazelwood, Bill Jia, Hsien-Hsin Lee, Andrey Malevich, Dheevatsa Mudigere, Mikhail Smelyanskiy, Liang Xiong, and Xuan Zhang. 2020. “[The Architectural Implications of Facebook’s DNN-Based Personalized Recommendation](/publications/architectural-implications-facebooks-dnn-based-personalized-recommendation)”. In . The 26th IEEE International Symposium on High-Performance Computer Architecture Udit Gupta, Carole Wu, Xiaodong Wang, Maxim Naumov, Brandon Reagen, David Brooks, Bradford Cottel, Kim Hazelwood, Bill Jia, Hsien-Hsin Lee, Andrey Malevich, Dheevatsa Mudigere, Mikhail Smelyanskiy, Liang Xiong, and Xuan Zhang. 2020. “[The Architectural Implications of Facebook’s DNN-Based Personalized Recommendation](/publications/architectural-implications-facebooks-dnn-based-personalized-recommendation)”. In . The 26th IEEE International Symposium on High-Performance Computer Architecture - add\_circle\_outline do\_not\_disturb\_on Abstract - [ picture\_as\_pdfThe Architectural Implica...](/sites/g/files/omnuum11281/files/vlsiarch/files/1906.03109.pdf) The widespread application of deep learning has changed the landscape of computation in the data center. In particular, personalized recommendation for content ranking is now largely accomplished leveraging deep neural networks. However, despite the... - [ picture\_as\_pdfThe Architectural Implica...](/sites/g/files/omnuum11281/files/vlsiarch/files/1906.03109.pdf) ### 2019 Paul Whatmough, Sae Lee, Marco Donato, Hsea Hsueh, Sam Xi, Udit Gupta, Lillian Pentecost, Glenn Ko, David Brooks, and Gu Wei. 2019. “[A 16nm 25mm2 SoC With a 54.5x Flexibility-Efficiency Range from Dual-Core Arm Cortex-A53 to EFPGA and Cache-Coherent Accelerators](/publications/16nm-25mm2-soc-545x-flexibility-efficiency-range-dual-core-arm-cortex-a53)”. Symposium on VLSI Circuits Paul Whatmough, Sae Lee, Marco Donato, Hsea Hsueh, Sam Xi, Udit Gupta, Lillian Pentecost, Glenn Ko, David Brooks, and Gu Wei. 2019. “[A 16nm 25mm2 SoC With a 54.5x Flexibility-Efficiency Range from Dual-Core Arm Cortex-A53 to EFPGA and Cache-Coherent Accelerators](/publications/16nm-25mm2-soc-545x-flexibility-efficiency-range-dual-core-arm-cortex-a53)”. Symposium on VLSI Circuits - add\_circle\_outline do\_not\_disturb\_on Abstract - [ descriptionPublisher's Version](https://ieeexplore.ieee.org/abstract/document/8778002/authors#authors) - [ picture\_as\_pdfA 16nm 25mm2 SoC with a 5...](/sites/g/files/omnuum11281/files/vlsiarch/files/smiv.pdf) This paper presents a 25mm^2 SoC in 16nm FinFET technology targeting flexible acceleration of compute intensive kernels in DNN, DSP and security algorithms. The SoC includes an always-on sub-system, a dual-core Arm A53 CPU cluster, an embedded FPGA array... - [ descriptionPublisher's Version](https://ieeexplore.ieee.org/abstract/document/8778002/authors#authors) - [ picture\_as\_pdfA 16nm 25mm2 SoC with a 5...](/sites/g/files/omnuum11281/files/vlsiarch/files/smiv.pdf) [ See all project publications arrow\_circle\_right ](https://prod-vlsiarch.drupalsites.harvard.edu/publications?f%5B0%5D=bibcite_reference_hwp_c_project123456%3A172593)