There is quite a bit involved with loading this data. You might want to refer to the code block as you read though this text.
The first for loop in the code below is going to loop through the entire input file or some number of samples that we specify when we call load_data(). I'm doing this because you might not have the RAM to load the entire dataset. You might get good results with as few as 10,000 examples; however, more is always better.
As we loop through the input file, line by line, we're doing several things at once:
- We're wrapping each French translation in a '\t' to start the phrase and a '\n' to end it. This corresponds to the <SOS> and <EOS> tags I used in the sequence-to-sequence diagram. This will allow us to ...
