About the Role
SEMRON's unique approach based on the proprietary CapRAM™ technology enables LLM inference while only transferring activations to/from memory dies. This results in a much lower total cost of operation due to multiple reasons, including much cheaper packaging compared to HBM.With CapRAM™, we are able to compute dot-products directly inside 3D-NAND-Flash. All remaining computations and data movements are performed on an SoC directly attached to this compute-capable memory. In this role, you will lead the development of the details of the digital architecture combining SRAM, DMA units, CPUs, Vector Processors, Routers and Chiplet Communication Units according to the specs already developed by SEMRON's team.
What you will do:
- Design and lead the implementation of the SoC for the chiplets of our Aloe hardware generation
- Analyze and evaluate trade-offs regarding the details of the microarchitectural design
- Closely collaborate with the system architecture, compiler, digital design and verification teams
What you should bring in:
- 5+ years of experience translating high-level requirements into detailed architectural direction
- 3+ years of experience building accelerators for ML workloads
- Willingness to collaborate across design, verification and software teams to keep execution moving
Helpful but not required:
- Deep knowledge of the details of serving modern LLMs in a datacenter setting
- Experience with dataflow architectures for ML workloads