VoxServe: Streaming-Centric Serving System for Speech Language Models
IntermediateKeisuke Kamahori, Wei-Tzu Lee et al.Jan 30arXiv
VoxServe is a new serving system that makes voice AIs respond fast and smoothly when streaming audio to users.
#Speech Language Models#streaming#Time-To-First-Audio