Speculative decoding can help AI chatbots improve throughput and reduce hardware demand by using a smaller model to draft tokens that a larger model validates.
The work relies in part on a transformer model, similar to the ones that power ChatGPT. Alex Huth (left), Shailee Jain (center) and Jerry Tang (right) prepare to collect brain activity data in the ...
Some results have been hidden because they may be inaccessible to you
Show inaccessible results