For sampling, we implemented a greedy sampler. Greedy samplers can get stuck repeating the same phrases over and over; for example, they may get stuck saying for the for the for the.... To prevent this, we could also implement a more random way of sampling words, maybe by making a weighted sampler based on the logits or probability distribution of the output.
There's more...
Get TensorFlow Machine Learning Cookbook - Second Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.