deeplearn@ML-RefVm-967342:~$ mkdir newsgroups deeplearn@ML-RefVm-967342:~$ cd newsgroups deeplearn@ML-RefVm-967342:~/newsgroups$ kaggle competitions download -c ml530-2022-fall-newsgroups Downloading ml530-2022-fall-newsgroups.zip to /home/deeplearn/newsgroups 45%|██████████████████████████████████████████▏ | 9.00M/19.8M [00:00<00:00, 65.3MB/s] 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 19.8M/19.8M [00:00<00:00, 109MB/s] deeplearn@ML-RefVm-967342:~/newsgroups$ wget http://nlp.stanford.edu/data/glove.6B.zip --2022-11-05 04:06:40-- http://nlp.stanford.edu/data/glove.6B.zip Resolving nlp.stanford.edu (nlp.stanford.edu)... 171.64.67.140 Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:80... connected. HTTP request sent, awaiting response... 302 Found Location: https://nlp.stanford.edu/data/glove.6B.zip [following] --2022-11-05 04:06:40-- https://nlp.stanford.edu/data/glove.6B.zip Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://downloads.cs.stanford.edu/nlp/data/glove.6B.zip [following] --2022-11-05 04:06:41-- https://downloads.cs.stanford.edu/nlp/data/glove.6B.zip Resolving downloads.cs.stanford.edu (downloads.cs.stanford.edu)... 171.64.64.22 Connecting to downloads.cs.stanford.edu (downloads.cs.stanford.edu)|171.64.64.22|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 862182613 (822M) [application/zip] Saving to: ‘glove.6B.zip’ glove.6B.zip 100%[============================================================>] 822.24M 5.02MB/s in 2m 39s 2022-11-05 04:09:20 (5.18 MB/s) - ‘glove.6B.zip’ saved [862182613/862182613] deeplearn@ML-RefVm-967342:~/newsgroups$ wget https://cross-entropy.net/ML530/newsgroups-vocabulary.py.txt --2022-11-05 04:09:30-- https://cross-entropy.net/ML530/newsgroups-vocabulary.py.txt Resolving cross-entropy.net (cross-entropy.net)... 107.180.57.14 Connecting to cross-entropy.net (cross-entropy.net)|107.180.57.14|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1971 (1.9K) [text/plain] Saving to: ‘newsgroups-vocabulary.py.txt’ newsgroups-vocabulary.py.txt 100%[============================================================>] 1.92K --.-KB/s in 0s 2022-11-05 04:09:30 (1.41 GB/s) - ‘newsgroups-vocabulary.py.txt’ saved [1971/1971] deeplearn@ML-RefVm-967342:~/newsgroups$ time python newsgroups-vocabulary.py.txt real 3m3.121s user 2m0.060s sys 0m2.070s deeplearn@ML-RefVm-967342:~/newsgroups$ wget https://cross-entropy.net/ML530/newsgroups-tensors.py.txt --2022-11-05 04:12:45-- https://cross-entropy.net/ML530/newsgroups-tensors.py.txt Resolving cross-entropy.net (cross-entropy.net)... 107.180.57.14 Connecting to cross-entropy.net (cross-entropy.net)|107.180.57.14|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1471 (1.4K) [text/plain] Saving to: ‘newsgroups-tensors.py.txt’ newsgroups-tensors.py.txt 100%[============================================================>] 1.44K --.-KB/s in 0s 2022-11-05 04:12:46 (1.05 GB/s) - ‘newsgroups-tensors.py.txt’ saved [1471/1471] deeplearn@ML-RefVm-967342:~/newsgroups$ time python newsgroups-tensors.py.txt real 0m40.602s user 0m40.058s sys 0m1.118s deeplearn@ML-RefVm-967342:~/newsgroups$ wget https://cross-entropy.net/ML530/newsgroups-search.py.txt --2022-11-05 04:13:42-- https://cross-entropy.net/ML530/newsgroups-search.py.txt Resolving cross-entropy.net (cross-entropy.net)... 107.180.57.14 Connecting to cross-entropy.net (cross-entropy.net)|107.180.57.14|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 5834 (5.7K) [text/plain] Saving to: ‘newsgroups-search.py.txt’ newsgroups-search.py.txt 100%[============================================================>] 5.70K --.-KB/s in 0s 2022-11-05 04:13:42 (3.02 GB/s) - ‘newsgroups-search.py.txt’ saved [5834/5834] deeplearn@ML-RefVm-967342:~/newsgroups$ time python newsgroups-search.py.txt 2022-11-05 04:14:19.818260: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-11-05 04:14:30.223468: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 10794 MB memory: -> device: 0, name: Tesla K80, pci bus id: 0001:00:00.0, compute capability: 3.7 Search space summary Default search space size: 9 embedding_type (Choice) {'default': 'uninitialized', 'conditions': [], 'values': ['uninitialized', 'initialized', 'frozen'], 'ordered': False} width (Choice) {'default': 64, 'conditions': [], 'values': [64, 128, 256, 512], 'ordered': True} regularization_function (Choice) {'default': 'none', 'conditions': [], 'values': ['none', 'l1', 'l2', 'l1_l2'], 'ordered': False} regularization_penalty (Choice) {'default': 0.0001, 'conditions': [], 'values': [0.0001, 1e-05], 'ordered': True} dropout (Float) {'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.5, 'step': 0.1, 'sampling': None} batch_size (Choice) {'default': 32, 'conditions': [], 'values': [32, 64, 128, 256, 512, 1024], 'ordered': True} optimizer (Choice) {'default': 'adam', 'conditions': [], 'values': ['adam', 'rmsprop', 'sgd'], 'ordered': False} learning_rate (Choice) {'default': 0.01, 'conditions': [], 'values': [0.01, 0.001, 0.0001], 'ordered': True} learning_rate_schedule (Choice) {'default': 'none', 'conditions': [], 'values': ['none', 'exponential', 'cosine'], 'ordered': False} Search: Running Trial #1 Value |Best Value So Far |Hyperparameter frozen |? |embedding_type 256 |? |width l1 |? |regularization_function 1e-05 |? |regularization_penalty 0.1 |? |dropout 256 |? |batch_size adam |? |optimizer 0.01 |? |learning_rate cosine |? |learning_rate_schedule 2 |? |tuner/epochs 0 |? |tuner/initial_epoch 3 |? |tuner/bracket 0 |? |tuner/round Epoch 1/2 59/59 [==============================] - 16s 34ms/step - loss: 2.6465 - accuracy: 0.3134 - val_loss: 2.4852 - val_accuracy: 0.4302 Epoch 2/2 59/59 [==============================] - 2s 30ms/step - loss: 2.2272 - accuracy: 0.4682 - val_loss: 2.0177 - val_accuracy: 0.5613 Trial 1 Complete [00h 00m 19s] val_accuracy: 0.5613383054733276 Best val_accuracy So Far: 0.5613383054733276 Total elapsed time: 00h 00m 19s Search: Running Trial #2 Value |Best Value So Far |Hyperparameter uninitialized |frozen |embedding_type 256 |256 |width l1_l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.1 |dropout 512 |256 |batch_size sgd |adam |optimizer 0.0001 |0.01 |learning_rate none |cosine |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 30/30 [==============================] - 3s 93ms/step - loss: 3.9756 - accuracy: 0.0667 - val_loss: 3.9162 - val_accuracy: 0.1052 Epoch 2/2 30/30 [==============================] - 3s 95ms/step - loss: 3.8643 - accuracy: 0.1216 - val_loss: 3.8071 - val_accuracy: 0.1503 Trial 2 Complete [00h 00m 06s] val_accuracy: 0.15029208362102509 Best val_accuracy So Far: 0.5613383054733276 Total elapsed time: 00h 00m 25s Search: Running Trial #3 Value |Best Value So Far |Hyperparameter initialized |frozen |embedding_type 256 |256 |width l1 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.1 |dropout 64 |256 |batch_size adam |adam |optimizer 0.001 |0.01 |learning_rate exponential |cosine |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 236/236 [==============================] - 5s 20ms/step - loss: 2.7576 - accuracy: 0.3010 - val_loss: 2.4792 - val_accuracy: 0.5088 Epoch 2/2 236/236 [==============================] - 5s 20ms/step - loss: 2.2017 - accuracy: 0.5793 - val_loss: 1.9397 - val_accuracy: 0.6474 Trial 3 Complete [00h 00m 10s] val_accuracy: 0.647371232509613 Best val_accuracy So Far: 0.647371232509613 Total elapsed time: 00h 00m 36s Search: Running Trial #4 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |256 |width none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.4 |dropout 512 |64 |batch_size adam |adam |optimizer 0.01 |0.001 |learning_rate exponential |exponential |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 30/30 [==============================] - 6s 175ms/step - loss: 2.3775 - accuracy: 0.4484 - val_loss: 1.4625 - val_accuracy: 0.7074 Epoch 2/2 30/30 [==============================] - 7s 221ms/step - loss: 0.8570 - accuracy: 0.8245 - val_loss: 0.6347 - val_accuracy: 0.8332 Trial 4 Complete [00h 00m 13s] val_accuracy: 0.833244800567627 Best val_accuracy So Far: 0.833244800567627 Total elapsed time: 00h 00m 48s Search: Running Trial #5 Value |Best Value So Far |Hyperparameter initialized |uninitialized |embedding_type 256 |512 |width l2 |none |regularization_function 0.0001 |0.0001 |regularization_penalty 0.2 |0.1 |dropout 256 |512 |batch_size adam |adam |optimizer 0.001 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 59/59 [==============================] - 4s 59ms/step - loss: 2.9016 - accuracy: 0.1993 - val_loss: 2.7994 - val_accuracy: 0.3749 Epoch 2/2 59/59 [==============================] - 4s 65ms/step - loss: 2.7019 - accuracy: 0.4529 - val_loss: 2.5984 - val_accuracy: 0.4865 Trial 5 Complete [00h 00m 08s] val_accuracy: 0.48645779490470886 Best val_accuracy So Far: 0.833244800567627 Total elapsed time: 00h 00m 56s Search: Running Trial #6 Value |Best Value So Far |Hyperparameter frozen |uninitialized |embedding_type 64 |512 |width l1_l2 |none |regularization_function 1e-05 |0.0001 |regularization_penalty 0 |0.1 |dropout 64 |512 |batch_size sgd |adam |optimizer 0.001 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 236/236 [==============================] - 2s 6ms/step - loss: 2.8423 - accuracy: 0.2520 - val_loss: 2.6973 - val_accuracy: 0.3691 Epoch 2/2 236/236 [==============================] - 1s 5ms/step - loss: 2.5967 - accuracy: 0.4168 - val_loss: 2.4942 - val_accuracy: 0.4689 Trial 6 Complete [00h 00m 03s] val_accuracy: 0.468932569026947 Best val_accuracy So Far: 0.833244800567627 Total elapsed time: 00h 00m 59s Search: Running Trial #7 Value |Best Value So Far |Hyperparameter uninitialized |uninitialized |embedding_type 256 |512 |width l2 |none |regularization_function 1e-05 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 64 |512 |batch_size sgd |adam |optimizer 0.0001 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 236/236 [==============================] - 4s 15ms/step - loss: 3.0159 - accuracy: 0.1805 - val_loss: 3.0032 - val_accuracy: 0.3335 Epoch 2/2 236/236 [==============================] - 4s 16ms/step - loss: 2.9899 - accuracy: 0.4443 - val_loss: 2.9778 - val_accuracy: 0.4371 Trial 7 Complete [00h 00m 08s] val_accuracy: 0.43706852197647095 Best val_accuracy So Far: 0.833244800567627 Total elapsed time: 00h 01m 07s Search: Running Trial #8 Value |Best Value So Far |Hyperparameter frozen |uninitialized |embedding_type 512 |512 |width l2 |none |regularization_function 1e-05 |0.0001 |regularization_penalty 0 |0.1 |dropout 1024 |512 |batch_size sgd |adam |optimizer 0.01 |0.01 |learning_rate none |exponential |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 15/15 [==============================] - 1s 63ms/step - loss: 2.8234 - accuracy: 0.2175 - val_loss: 2.6218 - val_accuracy: 0.3617 Epoch 2/2 15/15 [==============================] - 1s 84ms/step - loss: 2.4983 - accuracy: 0.4240 - val_loss: 2.3455 - val_accuracy: 0.4381 Trial 8 Complete [00h 00m 03s] val_accuracy: 0.438130646944046 Best val_accuracy So Far: 0.833244800567627 Total elapsed time: 00h 01m 10s Search: Running Trial #9 Value |Best Value So Far |Hyperparameter frozen |uninitialized |embedding_type 128 |512 |width l2 |none |regularization_function 0.0001 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 32 |512 |batch_size rmsprop |adam |optimizer 0.0001 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 471/471 [==============================] - 3s 6ms/step - loss: 2.9926 - accuracy: 0.0732 - val_loss: 2.9504 - val_accuracy: 0.1158 Epoch 2/2 471/471 [==============================] - 3s 6ms/step - loss: 2.9262 - accuracy: 0.1531 - val_loss: 2.9005 - val_accuracy: 0.2034 Trial 9 Complete [00h 00m 06s] val_accuracy: 0.20339883863925934 Best val_accuracy So Far: 0.833244800567627 Total elapsed time: 00h 01m 16s Search: Running Trial #10 Value |Best Value So Far |Hyperparameter initialized |uninitialized |embedding_type 512 |512 |width l1_l2 |none |regularization_function 0.0001 |0.0001 |regularization_penalty 0.5 |0.1 |dropout 128 |512 |batch_size sgd |adam |optimizer 0.01 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 118/118 [==============================] - 4s 33ms/step - loss: 1.9711 - accuracy: 0.5406 - val_loss: 1.4923 - val_accuracy: 0.6973 Epoch 2/2 118/118 [==============================] - 4s 34ms/step - loss: 0.8507 - accuracy: 0.8105 - val_loss: 0.6248 - val_accuracy: 0.8412 Trial 10 Complete [00h 00m 09s] val_accuracy: 0.8412108421325684 Best val_accuracy So Far: 0.8412108421325684 Total elapsed time: 00h 01m 25s Search: Running Trial #11 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |512 |width none |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.4 |0.5 |dropout 512 |128 |batch_size rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate exponential |cosine |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 30/30 [==============================] - 6s 176ms/step - loss: 2.9712 - accuracy: 0.1886 - val_loss: 2.9355 - val_accuracy: 0.3309 Epoch 2/2 30/30 [==============================] - 6s 185ms/step - loss: 2.8796 - accuracy: 0.4794 - val_loss: 2.8118 - val_accuracy: 0.5029 Trial 11 Complete [00h 00m 12s] val_accuracy: 0.5029208660125732 Best val_accuracy So Far: 0.8412108421325684 Total elapsed time: 00h 01m 37s Search: Running Trial #12 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 128 |512 |width l1_l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.2 |0.5 |dropout 32 |128 |batch_size rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate none |cosine |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 471/471 [==============================] - 7s 13ms/step - loss: 1.0489 - accuracy: 0.7265 - val_loss: 0.4437 - val_accuracy: 0.8768 Epoch 2/2 471/471 [==============================] - 6s 14ms/step - loss: 0.1951 - accuracy: 0.9519 - val_loss: 0.3680 - val_accuracy: 0.9007 Trial 12 Complete [00h 00m 13s] val_accuracy: 0.9006903767585754 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 01m 50s Search: Running Trial #13 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |128 |width l1_l2 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.5 |0.2 |dropout 64 |32 |batch_size sgd |rmsprop |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 236/236 [==============================] - 8s 30ms/step - loss: 2.6684 - accuracy: 0.4541 - val_loss: 2.3989 - val_accuracy: 0.6479 Epoch 2/2 236/236 [==============================] - 9s 37ms/step - loss: 2.2956 - accuracy: 0.7148 - val_loss: 2.3109 - val_accuracy: 0.7345 Trial 13 Complete [00h 00m 17s] val_accuracy: 0.7344662547111511 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 02m 08s Search: Running Trial #14 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |128 |width l1_l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 128 |32 |batch_size adam |rmsprop |optimizer 0.001 |0.01 |learning_rate cosine |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 118/118 [==============================] - 4s 34ms/step - loss: 2.8901 - accuracy: 0.2288 - val_loss: 2.8328 - val_accuracy: 0.3234 Epoch 2/2 118/118 [==============================] - 4s 34ms/step - loss: 2.6824 - accuracy: 0.4568 - val_loss: 2.5387 - val_accuracy: 0.5220 Trial 14 Complete [00h 00m 09s] val_accuracy: 0.5220392942428589 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 02m 16s Search: Running Trial #15 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 512 |128 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 1024 |32 |batch_size rmsprop |rmsprop |optimizer 0.0001 |0.01 |learning_rate cosine |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 15/15 [==============================] - 2s 115ms/step - loss: 3.0261 - accuracy: 0.0577 - val_loss: 3.0340 - val_accuracy: 0.0536 Epoch 2/2 15/15 [==============================] - 2s 112ms/step - loss: 3.0201 - accuracy: 0.0588 - val_loss: 3.0255 - val_accuracy: 0.0547 Trial 15 Complete [00h 00m 04s] val_accuracy: 0.05469994619488716 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 02m 21s Search: Running Trial #16 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |128 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.4 |0.2 |dropout 32 |32 |batch_size rmsprop |rmsprop |optimizer 0.001 |0.01 |learning_rate none |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 471/471 [==============================] - 7s 13ms/step - loss: 2.6174 - accuracy: 0.3751 - val_loss: 2.1789 - val_accuracy: 0.5852 Epoch 2/2 471/471 [==============================] - 6s 14ms/step - loss: 1.7755 - accuracy: 0.6676 - val_loss: 1.4555 - val_accuracy: 0.7238 Trial 16 Complete [00h 00m 13s] val_accuracy: 0.7238449454307556 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 02m 34s Search: Running Trial #17 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 512 |128 |width l2 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 64 |32 |batch_size rmsprop |rmsprop |optimizer 0.0001 |0.01 |learning_rate exponential |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 236/236 [==============================] - 3s 9ms/step - loss: 2.9886 - accuracy: 0.0676 - val_loss: 2.9638 - val_accuracy: 0.0797 Epoch 2/2 236/236 [==============================] - 2s 9ms/step - loss: 2.9425 - accuracy: 0.0983 - val_loss: 2.9256 - val_accuracy: 0.1248 Trial 17 Complete [00h 00m 05s] val_accuracy: 0.12480085343122482 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 02m 39s Search: Running Trial #18 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 256 |128 |width none |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.1 |0.2 |dropout 512 |32 |batch_size adam |rmsprop |optimizer 0.0001 |0.01 |learning_rate exponential |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 30/30 [==============================] - 4s 109ms/step - loss: 3.0115 - accuracy: 0.0457 - val_loss: 2.9996 - val_accuracy: 0.0457 Epoch 2/2 30/30 [==============================] - 3s 115ms/step - loss: 2.9916 - accuracy: 0.0527 - val_loss: 2.9821 - val_accuracy: 0.0510 Trial 18 Complete [00h 00m 07s] val_accuracy: 0.05098247528076172 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 02m 46s Search: Running Trial #19 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 64 |128 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.3 |0.2 |dropout 1024 |32 |batch_size adam |rmsprop |optimizer 0.001 |0.01 |learning_rate none |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 6/15 [===========>..................] - ETA: 0s - loss: 5.2610 - accuracy: 0.0498WARNING:tensorflow:Callback method `on_train_batch_en15/15 [==============================] - 2s 76ms/step - loss: 4.8937 - accuracy: 0.0629 - val_loss: 4.2633 - val_accuracy: 0.0871 Epoch 2/2 15/15 [==============================] - 1s 65ms/step - loss: 3.8642 - accuracy: 0.1035 - val_loss: 3.4693 - val_accuracy: 0.1078 Trial 19 Complete [00h 00m 03s] val_accuracy: 0.10780669003725052 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 02m 49s Search: Running Trial #20 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 128 |128 |width none |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.5 |0.2 |dropout 1024 |32 |batch_size adam |rmsprop |optimizer 0.001 |0.01 |learning_rate none |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 15/15 [==============================] - 2s 116ms/step - loss: 2.9894 - accuracy: 0.1862 - val_loss: 2.9810 - val_accuracy: 0.3447 Epoch 2/2 15/15 [==============================] - 2s 110ms/step - loss: 2.9717 - accuracy: 0.3721 - val_loss: 2.9622 - val_accuracy: 0.3930 Trial 20 Complete [00h 00m 04s] val_accuracy: 0.3929899036884308 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 02m 53s Search: Running Trial #21 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 128 |128 |width l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.1 |0.2 |dropout 256 |32 |batch_size sgd |rmsprop |optimizer 0.0001 |0.01 |learning_rate cosine |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 59/59 [==============================] - 4s 59ms/step - loss: 3.0043 - accuracy: 0.0483 - val_loss: 2.9976 - val_accuracy: 0.0489 Epoch 2/2 59/59 [==============================] - 4s 62ms/step - loss: 2.9843 - accuracy: 0.0643 - val_loss: 2.9714 - val_accuracy: 0.0775 Trial 21 Complete [00h 00m 08s] val_accuracy: 0.07753584533929825 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 03m 01s Search: Running Trial #22 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 256 |128 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.5 |0.2 |dropout 64 |32 |batch_size rmsprop |rmsprop |optimizer 0.0001 |0.01 |learning_rate none |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 236/236 [==============================] - 5s 20ms/step - loss: 2.9924 - accuracy: 0.0547 - val_loss: 2.9546 - val_accuracy: 0.0696 Epoch 2/2 236/236 [==============================] - 5s 20ms/step - loss: 2.9291 - accuracy: 0.1143 - val_loss: 2.9025 - val_accuracy: 0.1891 Trial 22 Complete [00h 00m 10s] val_accuracy: 0.1890600174665451 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 03m 11s Search: Running Trial #23 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 512 |128 |width none |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.3 |0.2 |dropout 1024 |32 |batch_size adam |rmsprop |optimizer 0.0001 |0.01 |learning_rate cosine |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 15/15 [==============================] - 2s 115ms/step - loss: 3.0091 - accuracy: 0.0433 - val_loss: 3.0063 - val_accuracy: 0.0430 Epoch 2/2 15/15 [==============================] - 2s 113ms/step - loss: 3.0037 - accuracy: 0.0438 - val_loss: 2.9997 - val_accuracy: 0.0446 Trial 23 Complete [00h 00m 04s] val_accuracy: 0.044609665870666504 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 03m 15s Search: Running Trial #24 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 256 |128 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0 |0.2 |dropout 512 |32 |batch_size rmsprop |rmsprop |optimizer 0.01 |0.01 |learning_rate cosine |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 6/30 [=====>........................] - ETA: 1s - loss: 7.7905 - accuracy: 0.0667WARNING:tensorflow:Callback method `on_train_batch_en30/30 [==============================] - 2s 59ms/step - loss: 4.8540 - accuracy: 0.1123 - val_loss: 3.2258 - val_accuracy: 0.1296 Epoch 2/2 30/30 [==============================] - 2s 65ms/step - loss: 4.2744 - accuracy: 0.1410 - val_loss: 3.2682 - val_accuracy: 0.1583 Trial 24 Complete [00h 00m 04s] val_accuracy: 0.1582580953836441 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 03m 19s Search: Running Trial #25 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |128 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 256 |32 |batch_size rmsprop |rmsprop |optimizer 0.0001 |0.01 |learning_rate exponential |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 59/59 [==============================] - 6s 92ms/step - loss: 20.8105 - accuracy: 0.0521 - val_loss: 18.6782 - val_accuracy: 0.0728 Epoch 2/2 59/59 [==============================] - 6s 94ms/step - loss: 16.8516 - accuracy: 0.0799 - val_loss: 15.0628 - val_accuracy: 0.0929 Trial 25 Complete [00h 00m 12s] val_accuracy: 0.09293680638074875 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 03m 31s Search: Running Trial #26 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |128 |width l1_l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.5 |0.2 |dropout 128 |32 |batch_size adam |rmsprop |optimizer 0.0001 |0.01 |learning_rate exponential |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 118/118 [==============================] - 6s 51ms/step - loss: 19.2525 - accuracy: 0.0673 - val_loss: 15.2222 - val_accuracy: 0.0924 Epoch 2/2 118/118 [==============================] - 6s 53ms/step - loss: 12.1143 - accuracy: 0.1010 - val_loss: 9.3274 - val_accuracy: 0.1115 Trial 26 Complete [00h 00m 13s] val_accuracy: 0.11152416467666626 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 03m 44s Search: Running Trial #27 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 256 |128 |width l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0 |0.2 |dropout 256 |32 |batch_size sgd |rmsprop |optimizer 0.001 |0.01 |learning_rate cosine |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 59/59 [==============================] - 2s 18ms/step - loss: 2.9509 - accuracy: 0.1075 - val_loss: 2.9252 - val_accuracy: 0.1498 Epoch 2/2 59/59 [==============================] - 1s 15ms/step - loss: 2.8740 - accuracy: 0.2467 - val_loss: 2.8256 - val_accuracy: 0.3101 Trial 27 Complete [00h 00m 03s] val_accuracy: 0.310143381357193 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 03m 47s Search: Running Trial #28 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 64 |128 |width none |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 256 |32 |batch_size sgd |rmsprop |optimizer 0.0001 |0.01 |learning_rate exponential |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 59/59 [==============================] - 4s 59ms/step - loss: 3.0160 - accuracy: 0.0526 - val_loss: 2.9944 - val_accuracy: 0.0489 Epoch 2/2 59/59 [==============================] - 4s 62ms/step - loss: 2.9827 - accuracy: 0.0578 - val_loss: 2.9673 - val_accuracy: 0.0643 Trial 28 Complete [00h 00m 08s] val_accuracy: 0.06425916403532028 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 03m 55s Search: Running Trial #29 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |128 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 64 |32 |batch_size sgd |rmsprop |optimizer 0.01 |0.01 |learning_rate none |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 236/236 [==============================] - 5s 20ms/step - loss: 1.2913 - accuracy: 0.6704 - val_loss: 0.5369 - val_accuracy: 0.8566 Epoch 2/2 236/236 [==============================] - 5s 20ms/step - loss: 0.2821 - accuracy: 0.9318 - val_loss: 0.3923 - val_accuracy: 0.8938 Trial 29 Complete [00h 00m 10s] val_accuracy: 0.8937864899635315 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 04m 06s Search: Running Trial #30 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 512 |128 |width none |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.3 |0.2 |dropout 512 |32 |batch_size adam |rmsprop |optimizer 0.001 |0.01 |learning_rate exponential |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 30/30 [==============================] - 2s 59ms/step - loss: 2.9638 - accuracy: 0.1033 - val_loss: 2.9145 - val_accuracy: 0.1949 Epoch 2/2 30/30 [==============================] - 2s 58ms/step - loss: 2.8863 - accuracy: 0.2200 - val_loss: 2.8461 - val_accuracy: 0.3192 Trial 30 Complete [00h 00m 04s] val_accuracy: 0.3191715478897095 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 04m 10s Search: Running Trial #31 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |128 |width l1_l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 128 |32 |batch_size adam |rmsprop |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 118/118 [==============================] - 6s 51ms/step - loss: 4.7479 - accuracy: 0.1473 - val_loss: 3.7110 - val_accuracy: 0.2034 Epoch 2/2 118/118 [==============================] - 6s 52ms/step - loss: 3.6372 - accuracy: 0.2603 - val_loss: 3.5736 - val_accuracy: 0.2889 Trial 31 Complete [00h 00m 13s] val_accuracy: 0.2889007031917572 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 04m 23s Search: Running Trial #32 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 64 |128 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.1 |0.2 |dropout 256 |32 |batch_size rmsprop |rmsprop |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 4/59 [=>............................] - ETA: 0s - loss: 3.1574 - accuracy: 0.0996 WARNING:tensorflow:Callback method `on_train_batch_e59/59 [==============================] - 2s 21ms/step - loss: 2.8210 - accuracy: 0.2530 - val_loss: 2.4623 - val_accuracy: 0.4052 Epoch 2/2 59/59 [==============================] - 1s 18ms/step - loss: 2.1608 - accuracy: 0.5147 - val_loss: 1.8947 - val_accuracy: 0.6022 Trial 32 Complete [00h 00m 03s] val_accuracy: 0.6022304892539978 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 04m 26s Search: Running Trial #33 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 128 |128 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.1 |0.2 |dropout 64 |32 |batch_size sgd |rmsprop |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 236/236 [==============================] - 3s 9ms/step - loss: 2.1939 - accuracy: 0.4257 - val_loss: 1.7247 - val_accuracy: 0.5836 Epoch 2/2 236/236 [==============================] - 2s 9ms/step - loss: 1.5463 - accuracy: 0.6093 - val_loss: 1.3998 - val_accuracy: 0.6410 Trial 33 Complete [00h 00m 05s] val_accuracy: 0.6409984230995178 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 04m 31s Search: Running Trial #34 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |128 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.1 |0.2 |dropout 1024 |32 |batch_size adam |rmsprop |optimizer 0.01 |0.01 |learning_rate cosine |none |learning_rate_schedule 2 |2 |tuner/epochs 0 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 0 |0 |tuner/round Epoch 1/2 15/15 [==============================] - 4s 213ms/step - loss: 2.7957 - accuracy: 0.3005 - val_loss: 2.6692 - val_accuracy: 0.4572 Epoch 2/2 15/15 [==============================] - 3s 227ms/step - loss: 2.3578 - accuracy: 0.5375 - val_loss: 2.0549 - val_accuracy: 0.6113 Trial 34 Complete [00h 00m 07s] val_accuracy: 0.6112586259841919 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 04m 38s Search: Running Trial #35 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 128 |128 |width l1_l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 32 |32 |batch_size rmsprop |rmsprop |optimizer 0.01 |0.01 |learning_rate none |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0011 |None |tuner/trial_id Epoch 3/4 471/471 [==============================] - 7s 13ms/step - loss: 1.0510 - accuracy: 0.7221 - val_loss: 0.4563 - val_accuracy: 0.8784 Epoch 4/4 471/471 [==============================] - 6s 14ms/step - loss: 0.1946 - accuracy: 0.9504 - val_loss: 0.3940 - val_accuracy: 0.8954 Trial 35 Complete [00h 00m 13s] val_accuracy: 0.8953797221183777 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 04m 51s Search: Running Trial #36 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |128 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 64 |32 |batch_size sgd |rmsprop |optimizer 0.01 |0.01 |learning_rate none |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0028 |None |tuner/trial_id Epoch 3/4 236/236 [==============================] - 5s 20ms/step - loss: 1.2895 - accuracy: 0.6690 - val_loss: 0.5455 - val_accuracy: 0.8508 Epoch 4/4 236/236 [==============================] - 5s 21ms/step - loss: 0.2779 - accuracy: 0.9323 - val_loss: 0.3815 - val_accuracy: 0.8970 Trial 36 Complete [00h 00m 10s] val_accuracy: 0.8969728946685791 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 05m 02s Search: Running Trial #37 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |128 |width l1_l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.5 |0.2 |dropout 128 |32 |batch_size sgd |rmsprop |optimizer 0.01 |0.01 |learning_rate cosine |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0009 |None |tuner/trial_id Epoch 3/4 118/118 [==============================] - 4s 33ms/step - loss: 1.9661 - accuracy: 0.5362 - val_loss: 1.4941 - val_accuracy: 0.6952 Epoch 4/4 118/118 [==============================] - 4s 35ms/step - loss: 0.8542 - accuracy: 0.8036 - val_loss: 0.6241 - val_accuracy: 0.8428 Trial 37 Complete [00h 00m 09s] val_accuracy: 0.8428040146827698 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 05m 10s Search: Running Trial #38 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |128 |width none |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.1 |0.2 |dropout 512 |32 |batch_size adam |rmsprop |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0003 |None |tuner/trial_id Epoch 3/4 30/30 [==============================] - 6s 175ms/step - loss: 2.3839 - accuracy: 0.4455 - val_loss: 1.4642 - val_accuracy: 0.7063 Epoch 4/4 30/30 [==============================] - 5s 181ms/step - loss: 0.8660 - accuracy: 0.8209 - val_loss: 0.6431 - val_accuracy: 0.8317 Trial 38 Complete [00h 00m 11s] val_accuracy: 0.8316516280174255 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 05m 22s Search: Running Trial #39 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |128 |width l1_l2 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.5 |0.2 |dropout 64 |32 |batch_size sgd |rmsprop |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0012 |None |tuner/trial_id Epoch 3/4 236/236 [==============================] - 8s 30ms/step - loss: 2.6630 - accuracy: 0.4521 - val_loss: 2.4138 - val_accuracy: 0.6474 Epoch 4/4 236/236 [==============================] - 7s 31ms/step - loss: 2.2958 - accuracy: 0.7070 - val_loss: 2.2874 - val_accuracy: 0.7339 Trial 39 Complete [00h 00m 15s] val_accuracy: 0.7339352369308472 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 05m 37s Search: Running Trial #40 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |128 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.4 |0.2 |dropout 32 |32 |batch_size rmsprop |rmsprop |optimizer 0.001 |0.01 |learning_rate none |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0015 |None |tuner/trial_id Epoch 3/4 471/471 [==============================] - 7s 13ms/step - loss: 2.6240 - accuracy: 0.3774 - val_loss: 2.1911 - val_accuracy: 0.5587 Epoch 4/4 471/471 [==============================] - 6s 13ms/step - loss: 1.7832 - accuracy: 0.6606 - val_loss: 1.4571 - val_accuracy: 0.7233 Trial 40 Complete [00h 00m 13s] val_accuracy: 0.7233138680458069 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 05m 50s Search: Running Trial #41 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 256 |128 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.4 |0.2 |dropout 64 |32 |batch_size adam |rmsprop |optimizer 0.001 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0002 |None |tuner/trial_id Epoch 3/4 236/236 [==============================] - 5s 20ms/step - loss: 2.7626 - accuracy: 0.2944 - val_loss: 2.4950 - val_accuracy: 0.4631 Epoch 4/4 236/236 [==============================] - 5s 20ms/step - loss: 2.2104 - accuracy: 0.5843 - val_loss: 1.9505 - val_accuracy: 0.6304 Trial 41 Complete [00h 00m 10s] val_accuracy: 0.6303770542144775 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 06m 01s Search: Running Trial #42 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 128 |128 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.1 |0.2 |dropout 64 |32 |batch_size sgd |rmsprop |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0032 |None |tuner/trial_id Epoch 3/4 236/236 [==============================] - 3s 9ms/step - loss: 2.1961 - accuracy: 0.4342 - val_loss: 1.7267 - val_accuracy: 0.5597 Epoch 4/4 236/236 [==============================] - 2s 9ms/step - loss: 1.5436 - accuracy: 0.6186 - val_loss: 1.4004 - val_accuracy: 0.6442 Trial 42 Complete [00h 00m 05s] val_accuracy: 0.6441848278045654 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 06m 05s Search: Running Trial #43 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |128 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.1 |0.2 |dropout 1024 |32 |batch_size adam |rmsprop |optimizer 0.01 |0.01 |learning_rate cosine |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0033 |None |tuner/trial_id Epoch 3/4 15/15 [==============================] - 4s 215ms/step - loss: 2.7962 - accuracy: 0.2651 - val_loss: 2.6673 - val_accuracy: 0.3707 Epoch 4/4 15/15 [==============================] - 3s 229ms/step - loss: 2.3646 - accuracy: 0.5060 - val_loss: 2.0598 - val_accuracy: 0.5836 Trial 43 Complete [00h 00m 07s] val_accuracy: 0.5836431384086609 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 06m 13s Search: Running Trial #44 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 64 |128 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.1 |0.2 |dropout 256 |32 |batch_size rmsprop |rmsprop |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0031 |None |tuner/trial_id Epoch 3/4 4/59 [=>............................] - ETA: 0s - loss: 3.1600 - accuracy: 0.0732 WARNING:tensorflow:Callback method `on_train_batch_e59/59 [==============================] - 2s 21ms/step - loss: 2.8298 - accuracy: 0.2585 - val_loss: 2.4617 - val_accuracy: 0.4052 Epoch 4/4 59/59 [==============================] - 1s 18ms/step - loss: 2.1521 - accuracy: 0.5204 - val_loss: 1.8808 - val_accuracy: 0.6059 Trial 44 Complete [00h 00m 03s] val_accuracy: 0.6059479713439941 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 06m 16s Search: Running Trial #45 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 256 |128 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.1 |0.2 |dropout 256 |32 |batch_size adam |rmsprop |optimizer 0.01 |0.01 |learning_rate cosine |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0000 |None |tuner/trial_id Epoch 3/4 59/59 [==============================] - 2s 31ms/step - loss: 2.6560 - accuracy: 0.3264 - val_loss: 2.4953 - val_accuracy: 0.4397 Epoch 4/4 59/59 [==============================] - 2s 28ms/step - loss: 2.2316 - accuracy: 0.4950 - val_loss: 2.0252 - val_accuracy: 0.5640 Trial 45 Complete [00h 00m 04s] val_accuracy: 0.5639936327934265 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 06m 20s Search: Running Trial #46 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |128 |width l1_l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 128 |32 |batch_size adam |rmsprop |optimizer 0.001 |0.01 |learning_rate cosine |none |learning_rate_schedule 4 |2 |tuner/epochs 2 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 1 |0 |tuner/round 0013 |None |tuner/trial_id Epoch 3/4 118/118 [==============================] - 4s 34ms/step - loss: 2.8833 - accuracy: 0.2462 - val_loss: 2.8285 - val_accuracy: 0.3521 Epoch 4/4 118/118 [==============================] - 4s 34ms/step - loss: 2.6739 - accuracy: 0.4519 - val_loss: 2.5300 - val_accuracy: 0.5082 Trial 46 Complete [00h 00m 09s] val_accuracy: 0.508231520652771 Best val_accuracy So Far: 0.9006903767585754 Total elapsed time: 00h 06m 29s Search: Running Trial #47 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |128 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.2 |0.2 |dropout 64 |32 |batch_size sgd |rmsprop |optimizer 0.01 |0.01 |learning_rate none |none |learning_rate_schedule 11 |2 |tuner/epochs 4 |0 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |0 |tuner/round 0035 |None |tuner/trial_id Epoch 5/11 236/236 [==============================] - 5s 20ms/step - loss: 1.2880 - accuracy: 0.6694 - val_loss: 0.5471 - val_accuracy: 0.8555 Epoch 6/11 236/236 [==============================] - 5s 20ms/step - loss: 0.2770 - accuracy: 0.9315 - val_loss: 0.4004 - val_accuracy: 0.8895 Epoch 7/11 236/236 [==============================] - 6s 24ms/step - loss: 0.0965 - accuracy: 0.9829 - val_loss: 0.3775 - val_accuracy: 0.8948 Epoch 8/11 236/236 [==============================] - 5s 22ms/step - loss: 0.0399 - accuracy: 0.9945 - val_loss: 0.3771 - val_accuracy: 0.9002 Epoch 9/11 236/236 [==============================] - 5s 20ms/step - loss: 0.0221 - accuracy: 0.9974 - val_loss: 0.3665 - val_accuracy: 0.9071 Epoch 10/11 236/236 [==============================] - 4s 19ms/step - loss: 0.0136 - accuracy: 0.9987 - val_loss: 0.3825 - val_accuracy: 0.9065 Epoch 11/11 236/236 [==============================] - 4s 19ms/step - loss: 0.0128 - accuracy: 0.9984 - val_loss: 0.3914 - val_accuracy: 0.9065 Trial 47 Complete [00h 00m 35s] val_accuracy: 0.9070631861686707 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 07m 04s Search: Running Trial #48 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 128 |512 |width l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.2 |dropout 32 |64 |batch_size rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate none |none |learning_rate_schedule 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |2 |tuner/round 0034 |0035 |tuner/trial_id Epoch 5/11 471/471 [==============================] - 7s 13ms/step - loss: 1.0491 - accuracy: 0.7233 - val_loss: 0.4664 - val_accuracy: 0.8699 Epoch 6/11 471/471 [==============================] - 6s 13ms/step - loss: 0.2007 - accuracy: 0.9508 - val_loss: 0.3761 - val_accuracy: 0.8991 Epoch 7/11 471/471 [==============================] - 6s 13ms/step - loss: 0.0573 - accuracy: 0.9888 - val_loss: 0.3963 - val_accuracy: 0.8938 Epoch 8/11 471/471 [==============================] - 6s 13ms/step - loss: 0.0299 - accuracy: 0.9950 - val_loss: 0.4051 - val_accuracy: 0.8991 Epoch 9/11 471/471 [==============================] - 7s 14ms/step - loss: 0.0147 - accuracy: 0.9981 - val_loss: 0.4126 - val_accuracy: 0.9049 Epoch 10/11 471/471 [==============================] - 6s 13ms/step - loss: 0.0127 - accuracy: 0.9985 - val_loss: 0.4386 - val_accuracy: 0.8964 Epoch 11/11 471/471 [==============================] - 6s 13ms/step - loss: 0.0104 - accuracy: 0.9986 - val_loss: 0.4580 - val_accuracy: 0.8927 Trial 48 Complete [00h 00m 44s] val_accuracy: 0.9049389362335205 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 07m 48s Search: Running Trial #49 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |512 |width l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.2 |dropout 128 |64 |batch_size sgd |sgd |optimizer 0.01 |0.01 |learning_rate cosine |none |learning_rate_schedule 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |2 |tuner/round 0036 |0035 |tuner/trial_id Epoch 5/11 118/118 [==============================] - 4s 33ms/step - loss: 1.9573 - accuracy: 0.5407 - val_loss: 1.4787 - val_accuracy: 0.7021 Epoch 6/11 118/118 [==============================] - 4s 35ms/step - loss: 0.8515 - accuracy: 0.8043 - val_loss: 0.6199 - val_accuracy: 0.8471 Epoch 7/11 118/118 [==============================] - 4s 33ms/step - loss: 0.4130 - accuracy: 0.9171 - val_loss: 0.5549 - val_accuracy: 0.8603 Epoch 8/11 118/118 [==============================] - 4s 34ms/step - loss: 0.3124 - accuracy: 0.9292 - val_loss: 0.4180 - val_accuracy: 0.8922 Epoch 9/11 118/118 [==============================] - 4s 33ms/step - loss: 0.1515 - accuracy: 0.9720 - val_loss: 0.3758 - val_accuracy: 0.8975 Epoch 10/11 118/118 [==============================] - 4s 32ms/step - loss: 0.0958 - accuracy: 0.9885 - val_loss: 0.3670 - val_accuracy: 0.9002 Epoch 11/11 118/118 [==============================] - 4s 35ms/step - loss: 0.0788 - accuracy: 0.9926 - val_loss: 0.3645 - val_accuracy: 0.9007 Trial 49 Complete [00h 00m 28s] val_accuracy: 0.9006903767585754 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 08m 16s Search: Running Trial #50 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |512 |width none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.2 |dropout 512 |64 |batch_size adam |sgd |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 2 |2 |tuner/round 0037 |0035 |tuner/trial_id Epoch 5/11 30/30 [==============================] - 6s 177ms/step - loss: 2.4011 - accuracy: 0.4466 - val_loss: 1.4858 - val_accuracy: 0.6952 Epoch 6/11 30/30 [==============================] - 6s 186ms/step - loss: 0.8773 - accuracy: 0.8162 - val_loss: 0.6436 - val_accuracy: 0.8338 Epoch 7/11 30/30 [==============================] - 6s 212ms/step - loss: 0.3426 - accuracy: 0.9274 - val_loss: 0.4648 - val_accuracy: 0.8715 Epoch 8/11 30/30 [==============================] - 5s 180ms/step - loss: 0.1757 - accuracy: 0.9695 - val_loss: 0.3988 - val_accuracy: 0.8911 Epoch 9/11 30/30 [==============================] - 5s 183ms/step - loss: 0.1036 - accuracy: 0.9869 - val_loss: 0.3764 - val_accuracy: 0.8986 Epoch 10/11 30/30 [==============================] - 5s 175ms/step - loss: 0.0654 - accuracy: 0.9939 - val_loss: 0.3689 - val_accuracy: 0.9018 Epoch 11/11 30/30 [==============================] - 5s 172ms/step - loss: 0.0444 - accuracy: 0.9969 - val_loss: 0.3645 - val_accuracy: 0.9028 Trial 50 Complete [00h 00m 39s] val_accuracy: 0.9028146862983704 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 08m 55s Search: Running Trial #51 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |512 |width l1 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.2 |dropout 64 |64 |batch_size sgd |sgd |optimizer 0.01 |0.01 |learning_rate none |none |learning_rate_schedule 32 |11 |tuner/epochs 11 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 3 |2 |tuner/round 0046 |0035 |tuner/trial_id Epoch 12/32 236/236 [==============================] - 5s 20ms/step - loss: 1.2972 - accuracy: 0.6637 - val_loss: 0.5427 - val_accuracy: 0.8561 Epoch 13/32 236/236 [==============================] - 5s 20ms/step - loss: 0.2779 - accuracy: 0.9343 - val_loss: 0.3850 - val_accuracy: 0.8943 Epoch 14/32 236/236 [==============================] - 5s 20ms/step - loss: 0.0949 - accuracy: 0.9816 - val_loss: 0.3772 - val_accuracy: 0.9012 Epoch 15/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0415 - accuracy: 0.9944 - val_loss: 0.3726 - val_accuracy: 0.8986 Epoch 16/32 236/236 [==============================] - 5s 20ms/step - loss: 0.0233 - accuracy: 0.9973 - val_loss: 0.3861 - val_accuracy: 0.9023 Epoch 17/32 236/236 [==============================] - 5s 20ms/step - loss: 0.0152 - accuracy: 0.9983 - val_loss: 0.3968 - val_accuracy: 0.9033 Epoch 18/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0126 - accuracy: 0.9984 - val_loss: 0.3933 - val_accuracy: 0.9033 Epoch 19/32 236/236 [==============================] - 5s 20ms/step - loss: 0.0124 - accuracy: 0.9986 - val_loss: 0.4020 - val_accuracy: 0.9049 Epoch 20/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0080 - accuracy: 0.9991 - val_loss: 0.4144 - val_accuracy: 0.9023 Epoch 21/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0091 - accuracy: 0.9989 - val_loss: 0.4460 - val_accuracy: 0.9039 Epoch 22/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0075 - accuracy: 0.9991 - val_loss: 0.4286 - val_accuracy: 0.9028 Epoch 23/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0069 - accuracy: 0.9990 - val_loss: 0.4555 - val_accuracy: 0.8986 Epoch 24/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0054 - accuracy: 0.9994 - val_loss: 0.4408 - val_accuracy: 0.9049 Epoch 25/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0081 - accuracy: 0.9987 - val_loss: 0.4655 - val_accuracy: 0.9012 Epoch 26/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0087 - accuracy: 0.9985 - val_loss: 0.4823 - val_accuracy: 0.8975 Epoch 27/32 236/236 [==============================] - 5s 20ms/step - loss: 0.0062 - accuracy: 0.9992 - val_loss: 0.4599 - val_accuracy: 0.9065 Epoch 28/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0065 - accuracy: 0.9991 - val_loss: 0.5019 - val_accuracy: 0.8943 Epoch 29/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0074 - accuracy: 0.9990 - val_loss: 0.4930 - val_accuracy: 0.9002 Epoch 30/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0067 - accuracy: 0.9989 - val_loss: 0.4977 - val_accuracy: 0.9002 Epoch 31/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0050 - accuracy: 0.9993 - val_loss: 0.4808 - val_accuracy: 0.9044 Epoch 32/32 236/236 [==============================] - 4s 19ms/step - loss: 0.0056 - accuracy: 0.9992 - val_loss: 0.4940 - val_accuracy: 0.9060 Trial 51 Complete [00h 01m 36s] val_accuracy: 0.9065321087837219 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 10m 31s Search: Running Trial #52 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 128 |512 |width l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.2 |0.2 |dropout 32 |64 |batch_size rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate none |none |learning_rate_schedule 32 |11 |tuner/epochs 11 |4 |tuner/initial_epoch 3 |3 |tuner/bracket 3 |2 |tuner/round 0047 |0035 |tuner/trial_id Epoch 12/32 471/471 [==============================] - 7s 13ms/step - loss: 1.0471 - accuracy: 0.7291 - val_loss: 0.4676 - val_accuracy: 0.8773 Epoch 13/32 471/471 [==============================] - 6s 13ms/step - loss: 0.2001 - accuracy: 0.9514 - val_loss: 0.3698 - val_accuracy: 0.9002 Epoch 14/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0614 - accuracy: 0.9882 - val_loss: 0.4027 - val_accuracy: 0.8970 Epoch 15/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0280 - accuracy: 0.9950 - val_loss: 0.4099 - val_accuracy: 0.9002 Epoch 16/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0156 - accuracy: 0.9985 - val_loss: 0.4055 - val_accuracy: 0.8986 Epoch 17/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0129 - accuracy: 0.9983 - val_loss: 0.4298 - val_accuracy: 0.8996 Epoch 18/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0121 - accuracy: 0.9984 - val_loss: 0.4354 - val_accuracy: 0.8996 Epoch 19/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0121 - accuracy: 0.9981 - val_loss: 0.4736 - val_accuracy: 0.8948 Epoch 20/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0105 - accuracy: 0.9987 - val_loss: 0.5293 - val_accuracy: 0.8927 Epoch 21/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0175 - accuracy: 0.9969 - val_loss: 0.5173 - val_accuracy: 0.8975 Epoch 22/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0124 - accuracy: 0.9978 - val_loss: 0.5625 - val_accuracy: 0.8938 Epoch 23/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0108 - accuracy: 0.9985 - val_loss: 0.5460 - val_accuracy: 0.8970 Epoch 24/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0088 - accuracy: 0.9989 - val_loss: 0.5490 - val_accuracy: 0.8943 Epoch 25/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0069 - accuracy: 0.9992 - val_loss: 0.5377 - val_accuracy: 0.8948 Epoch 26/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0170 - accuracy: 0.9970 - val_loss: 0.6604 - val_accuracy: 0.8890 Epoch 27/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0220 - accuracy: 0.9967 - val_loss: 0.6359 - val_accuracy: 0.8922 Epoch 28/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0099 - accuracy: 0.9978 - val_loss: 0.7242 - val_accuracy: 0.8832 Epoch 29/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0133 - accuracy: 0.9975 - val_loss: 0.6883 - val_accuracy: 0.8848 Epoch 30/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0065 - accuracy: 0.9988 - val_loss: 0.6909 - val_accuracy: 0.8895 Epoch 31/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0064 - accuracy: 0.9989 - val_loss: 0.6795 - val_accuracy: 0.8911 Epoch 32/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0074 - accuracy: 0.9990 - val_loss: 0.6885 - val_accuracy: 0.8933 Trial 52 Complete [00h 02m 07s] val_accuracy: 0.9001592993736267 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 12m 38s Search: Running Trial #53 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 128 |512 |width l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.2 |dropout 64 |64 |batch_size sgd |sgd |optimizer 0.0001 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 236/236 [==============================] - 3s 9ms/step - loss: 2.9884 - accuracy: 0.0585 - val_loss: 2.9636 - val_accuracy: 0.0913 Epoch 2/4 236/236 [==============================] - 2s 9ms/step - loss: 2.9492 - accuracy: 0.1285 - val_loss: 2.9304 - val_accuracy: 0.1503 Epoch 3/4 236/236 [==============================] - 2s 8ms/step - loss: 2.9179 - accuracy: 0.1741 - val_loss: 2.9012 - val_accuracy: 0.1992 Epoch 4/4 236/236 [==============================] - 2s 8ms/step - loss: 2.8911 - accuracy: 0.2181 - val_loss: 2.8748 - val_accuracy: 0.2395 Trial 53 Complete [00h 00m 09s] val_accuracy: 0.2395114153623581 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 12m 47s Search: Running Trial #54 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 256 |512 |width l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.2 |dropout 256 |64 |batch_size rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate cosine |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 59/59 [==============================] - 4s 59ms/step - loss: 2.9376 - accuracy: 0.1530 - val_loss: 2.9026 - val_accuracy: 0.2469 Epoch 2/4 59/59 [==============================] - 4s 71ms/step - loss: 2.8155 - accuracy: 0.3293 - val_loss: 2.7407 - val_accuracy: 0.4344 Epoch 3/4 59/59 [==============================] - 4s 62ms/step - loss: 2.7057 - accuracy: 0.4739 - val_loss: 2.7002 - val_accuracy: 0.4716 Epoch 4/4 59/59 [==============================] - 4s 60ms/step - loss: 2.5858 - accuracy: 0.5024 - val_loss: 2.4756 - val_accuracy: 0.5534 Trial 54 Complete [00h 00m 16s] val_accuracy: 0.5533722639083862 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 13m 03s Search: Running Trial #55 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |512 |width l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.5 |0.2 |dropout 512 |64 |batch_size rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate cosine |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 30/30 [==============================] - 6s 171ms/step - loss: 3.0366 - accuracy: 0.2705 - val_loss: 3.0223 - val_accuracy: 0.3654 Epoch 2/4 30/30 [==============================] - 5s 174ms/step - loss: 2.9758 - accuracy: 0.4757 - val_loss: 2.9310 - val_accuracy: 0.5279 Epoch 3/4 30/30 [==============================] - 5s 162ms/step - loss: 2.9072 - accuracy: 0.5661 - val_loss: 2.9037 - val_accuracy: 0.5220 Epoch 4/4 30/30 [==============================] - 5s 181ms/step - loss: 2.8197 - accuracy: 0.5562 - val_loss: 2.7305 - val_accuracy: 0.5433 Trial 55 Complete [00h 00m 21s] val_accuracy: 0.5432819724082947 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 13m 24s Search: Running Trial #56 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 256 |512 |width l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.3 |0.2 |dropout 512 |64 |batch_size rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate none |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 30/30 [==============================] - 3s 92ms/step - loss: 2.6561 - accuracy: 0.3815 - val_loss: 2.0986 - val_accuracy: 0.6102 Epoch 2/4 30/30 [==============================] - 3s 94ms/step - loss: 1.6730 - accuracy: 0.7168 - val_loss: 1.4076 - val_accuracy: 0.7477 Epoch 3/4 30/30 [==============================] - 3s 94ms/step - loss: 1.1694 - accuracy: 0.8293 - val_loss: 1.1733 - val_accuracy: 0.8099 Epoch 4/4 30/30 [==============================] - 3s 93ms/step - loss: 0.9547 - accuracy: 0.8865 - val_loss: 1.0576 - val_accuracy: 0.8348 Trial 56 Complete [00h 00m 12s] val_accuracy: 0.8348380327224731 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 13m 36s Search: Running Trial #57 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |512 |width l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.2 |dropout 32 |64 |batch_size sgd |sgd |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 471/471 [==============================] - 7s 13ms/step - loss: 1.0403 - accuracy: 0.7289 - val_loss: 0.4813 - val_accuracy: 0.8593 Epoch 2/4 471/471 [==============================] - 6s 13ms/step - loss: 0.1926 - accuracy: 0.9523 - val_loss: 0.3889 - val_accuracy: 0.8948 Epoch 3/4 471/471 [==============================] - 6s 14ms/step - loss: 0.0598 - accuracy: 0.9887 - val_loss: 0.3721 - val_accuracy: 0.9028 Epoch 4/4 471/471 [==============================] - 6s 13ms/step - loss: 0.0248 - accuracy: 0.9958 - val_loss: 0.3716 - val_accuracy: 0.9028 Trial 57 Complete [00h 00m 26s] val_accuracy: 0.9028146862983704 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 14m 02s Search: Running Trial #58 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 64 |512 |width l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.1 |0.2 |dropout 128 |64 |batch_size sgd |sgd |optimizer 0.0001 |0.01 |learning_rate none |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 6/118 [>.............................] - ETA: 1s - loss: 3.0037 - accuracy: 0.0612 WARNING:tensorflow:Callback method `on_train_batch118/118 [==============================] - 2s 12ms/step - loss: 3.0011 - accuracy: 0.0930 - val_loss: 2.9979 - val_accuracy: 0.1264 Epoch 2/4 118/118 [==============================] - 1s 11ms/step - loss: 2.9945 - accuracy: 0.2076 - val_loss: 2.9915 - val_accuracy: 0.2650 Epoch 3/4 118/118 [==============================] - 1s 11ms/step - loss: 2.9875 - accuracy: 0.3404 - val_loss: 2.9845 - val_accuracy: 0.3776 Epoch 4/4 118/118 [==============================] - 1s 11ms/step - loss: 2.9797 - accuracy: 0.4450 - val_loss: 2.9765 - val_accuracy: 0.4334 Trial 58 Complete [00h 00m 06s] val_accuracy: 0.4333510398864746 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 14m 08s Search: Running Trial #59 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |512 |width l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0 |0.2 |dropout 1024 |64 |batch_size rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate cosine |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 6/15 [===========>..................] - ETA: 1s - loss: 3.6542 - accuracy: 0.0537WARNING:tensorflow:Callback method `on_train_batch_en15/15 [==============================] - 4s 214ms/step - loss: 3.6454 - accuracy: 0.0630 - val_loss: 3.6363 - val_accuracy: 0.0627 Epoch 2/4 15/15 [==============================] - 3s 208ms/step - loss: 3.6142 - accuracy: 0.0910 - val_loss: 3.5931 - val_accuracy: 0.1030 Epoch 3/4 15/15 [==============================] - 3s 209ms/step - loss: 3.5859 - accuracy: 0.1234 - val_loss: 3.5830 - val_accuracy: 0.1147 Epoch 4/4 15/15 [==============================] - 5s 323ms/step - loss: 3.5602 - accuracy: 0.1595 - val_loss: 3.5355 - val_accuracy: 0.1784 Trial 59 Complete [00h 00m 15s] val_accuracy: 0.17843866348266602 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 14m 23s Search: Running Trial #60 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 64 |512 |width none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.2 |dropout 512 |64 |batch_size rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate cosine |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 30/30 [==============================] - 2s 41ms/step - loss: 2.8980 - accuracy: 0.3407 - val_loss: 2.8262 - val_accuracy: 0.4578 Epoch 2/4 30/30 [==============================] - 1s 36ms/step - loss: 2.4631 - accuracy: 0.5882 - val_loss: 2.1127 - val_accuracy: 0.6394 Epoch 3/4 30/30 [==============================] - 1s 37ms/step - loss: 1.9306 - accuracy: 0.7023 - val_loss: 1.9312 - val_accuracy: 0.6745 Epoch 4/4 30/30 [==============================] - 1s 37ms/step - loss: 1.4798 - accuracy: 0.7694 - val_loss: 1.2075 - val_accuracy: 0.7764 Trial 60 Complete [00h 00m 05s] val_accuracy: 0.7764205932617188 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 14m 28s Search: Running Trial #61 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 256 |512 |width none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.2 |dropout 128 |64 |batch_size rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate cosine |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 118/118 [==============================] - 4s 29ms/step - loss: 2.9923 - accuracy: 0.1074 - val_loss: 2.9910 - val_accuracy: 0.1407 Epoch 2/4 118/118 [==============================] - 4s 31ms/step - loss: 2.9859 - accuracy: 0.2254 - val_loss: 2.9823 - val_accuracy: 0.2852 Epoch 3/4 118/118 [==============================] - 3s 29ms/step - loss: 2.9796 - accuracy: 0.3200 - val_loss: 2.9802 - val_accuracy: 0.3054 Epoch 4/4 118/118 [==============================] - 3s 28ms/step - loss: 2.9730 - accuracy: 0.3673 - val_loss: 2.9678 - val_accuracy: 0.4004 Trial 61 Complete [00h 00m 15s] val_accuracy: 0.40042486786842346 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 14m 43s Search: Running Trial #62 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 128 |512 |width l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.5 |0.2 |dropout 256 |64 |batch_size sgd |sgd |optimizer 0.0001 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 59/59 [==============================] - 4s 59ms/step - loss: 3.0038 - accuracy: 0.0540 - val_loss: 2.9954 - val_accuracy: 0.0457 Epoch 2/4 59/59 [==============================] - 4s 63ms/step - loss: 2.9783 - accuracy: 0.0590 - val_loss: 2.9722 - val_accuracy: 0.0478 Epoch 3/4 59/59 [==============================] - 4s 75ms/step - loss: 2.9582 - accuracy: 0.0710 - val_loss: 2.9527 - val_accuracy: 0.0621 Epoch 4/4 59/59 [==============================] - 4s 72ms/step - loss: 2.9403 - accuracy: 0.0991 - val_loss: 2.9353 - val_accuracy: 0.0844 Trial 62 Complete [00h 00m 16s] val_accuracy: 0.0844397246837616 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 14m 59s Search: Running Trial #63 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 256 |512 |width l1_l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.2 |0.2 |dropout 512 |64 |batch_size adam |sgd |optimizer 0.001 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 30/30 [==============================] - 4s 109ms/step - loss: 2.9457 - accuracy: 0.1416 - val_loss: 2.8783 - val_accuracy: 0.2533 Epoch 2/4 30/30 [==============================] - 4s 133ms/step - loss: 2.8254 - accuracy: 0.3401 - val_loss: 2.7659 - val_accuracy: 0.4201 Epoch 3/4 30/30 [==============================] - 3s 112ms/step - loss: 2.7083 - accuracy: 0.4558 - val_loss: 2.6487 - val_accuracy: 0.4950 Epoch 4/4 30/30 [==============================] - 3s 116ms/step - loss: 2.5854 - accuracy: 0.5240 - val_loss: 2.5287 - val_accuracy: 0.5343 Trial 63 Complete [00h 00m 15s] val_accuracy: 0.5342538356781006 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 15m 14s Search: Running Trial #64 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |512 |width l2 |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.2 |dropout 512 |64 |batch_size rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 30/30 [==============================] - 6s 173ms/step - loss: 3.0241 - accuracy: 0.3287 - val_loss: 2.9749 - val_accuracy: 0.4626 Epoch 2/4 30/30 [==============================] - 5s 178ms/step - loss: 2.9147 - accuracy: 0.5136 - val_loss: 2.8449 - val_accuracy: 0.5109 Epoch 3/4 30/30 [==============================] - 5s 175ms/step - loss: 2.7576 - accuracy: 0.5466 - val_loss: 2.6723 - val_accuracy: 0.5279 Epoch 4/4 30/30 [==============================] - 5s 175ms/step - loss: 2.5736 - accuracy: 0.5822 - val_loss: 2.4951 - val_accuracy: 0.5762 Trial 64 Complete [00h 00m 22s] val_accuracy: 0.5762081742286682 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 15m 36s Search: Running Trial #65 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 512 |512 |width none |l1 |regularization_function 1e-05 |1e-05 |regularization_penalty 0.4 |0.2 |dropout 256 |64 |batch_size sgd |sgd |optimizer 0.0001 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 59/59 [==============================] - 2s 30ms/step - loss: 3.0152 - accuracy: 0.0415 - val_loss: 3.0010 - val_accuracy: 0.0388 Epoch 2/4 59/59 [==============================] - 2s 29ms/step - loss: 2.9908 - accuracy: 0.0542 - val_loss: 2.9806 - val_accuracy: 0.0579 Epoch 3/4 59/59 [==============================] - 2s 30ms/step - loss: 2.9729 - accuracy: 0.0701 - val_loss: 2.9650 - val_accuracy: 0.0871 Epoch 4/4 59/59 [==============================] - 2s 28ms/step - loss: 2.9589 - accuracy: 0.0883 - val_loss: 2.9518 - val_accuracy: 0.1009 Trial 65 Complete [00h 00m 08s] val_accuracy: 0.10090281814336777 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 15m 43s Search: Running Trial #66 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 256 |512 |width none |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.4 |0.2 |dropout 1024 |64 |batch_size rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate exponential |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 15/15 [==============================] - 2s 142ms/step - loss: 3.0042 - accuracy: 0.0513 - val_loss: 2.9921 - val_accuracy: 0.0542 Epoch 2/4 15/15 [==============================] - 2s 114ms/step - loss: 2.9965 - accuracy: 0.0562 - val_loss: 2.9850 - val_accuracy: 0.0584 Epoch 3/4 15/15 [==============================] - 2s 107ms/step - loss: 2.9885 - accuracy: 0.0606 - val_loss: 2.9789 - val_accuracy: 0.0621 Epoch 4/4 15/15 [==============================] - 2s 107ms/step - loss: 2.9825 - accuracy: 0.0642 - val_loss: 2.9735 - val_accuracy: 0.0653 Trial 66 Complete [00h 00m 08s] val_accuracy: 0.06532129645347595 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 15m 51s Search: Running Trial #67 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 64 |512 |width l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0 |0.2 |dropout 256 |64 |batch_size adam |sgd |optimizer 0.01 |0.01 |learning_rate cosine |none |learning_rate_schedule 4 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 0 |2 |tuner/round Epoch 1/4 5/59 [=>............................] - ETA: 0s - loss: 4.3481 - accuracy: 0.0586 WARNING:tensorflow:Callback method `on_train_batch_e59/59 [==============================] - 2s 17ms/step - loss: 3.2504 - accuracy: 0.1175 - val_loss: 2.9383 - val_accuracy: 0.1434 Epoch 2/4 59/59 [==============================] - 1s 18ms/step - loss: 3.0480 - accuracy: 0.1603 - val_loss: 2.8235 - val_accuracy: 0.1869 Epoch 3/4 59/59 [==============================] - 1s 16ms/step - loss: 2.7600 - accuracy: 0.2057 - val_loss: 2.7185 - val_accuracy: 0.2045 Epoch 4/4 59/59 [==============================] - 1s 15ms/step - loss: 2.8997 - accuracy: 0.2391 - val_loss: 2.7357 - val_accuracy: 0.2581 Trial 67 Complete [00h 00m 05s] val_accuracy: 0.2580987811088562 Best val_accuracy So Far: 0.9070631861686707 Total elapsed time: 00h 15m 56s Search: Running Trial #68 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |512 |width l1_l2 |l1 |regularization_function 0.0001 |1e-05 |regularization_penalty 0.1 |0.2 |dropout 32 |64 |batch_size sgd |sgd |optimizer 0.01 |0.01 |learning_rate exponential |none |learning_rate_schedule 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 2 |3 |tuner/bracket 1 |2 |tuner/round 0056 |0035 |tuner/trial_id Epoch 5/11 471/471 [==============================] - 7s 13ms/step - loss: 1.0414 - accuracy: 0.7282 - val_loss: 0.4570 - val_accuracy: 0.8747 Epoch 6/11 471/471 [==============================] - 7s 14ms/step - loss: 0.1914 - accuracy: 0.9531 - val_loss: 0.3829 - val_accuracy: 0.9002 Epoch 7/11 471/471 [==============================] - 6s 14ms/step - loss: 0.0591 - accuracy: 0.9889 - val_loss: 0.3783 - val_accuracy: 0.9012 Epoch 8/11 471/471 [==============================] - 6s 14ms/step - loss: 0.0232 - accuracy: 0.9964 - val_loss: 0.3706 - val_accuracy: 0.9081 Epoch 9/11 471/471 [==============================] - 6s 13ms/step - loss: 0.0171 - accuracy: 0.9979 - val_loss: 0.3864 - val_accuracy: 0.9039 Epoch 10/11 471/471 [==============================] - 6s 13ms/step - loss: 0.0116 - accuracy: 0.9987 - val_loss: 0.3957 - val_accuracy: 0.9044 Epoch 11/11 471/471 [==============================] - 6s 13ms/step - loss: 0.0099 - accuracy: 0.9987 - val_loss: 0.4177 - val_accuracy: 0.9007 Trial 68 Complete [00h 00m 44s] val_accuracy: 0.9081253409385681 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 16m 40s Search: Running Trial #69 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 256 |512 |width l2 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 512 |32 |batch_size rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate none |exponential |learning_rate_schedule 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 2 |2 |tuner/bracket 1 |1 |tuner/round 0055 |0056 |tuner/trial_id Epoch 5/11 30/30 [==============================] - 3s 92ms/step - loss: 2.6632 - accuracy: 0.4096 - val_loss: 2.1107 - val_accuracy: 0.6187 Epoch 6/11 30/30 [==============================] - 3s 95ms/step - loss: 1.6818 - accuracy: 0.7193 - val_loss: 1.4181 - val_accuracy: 0.7499 Epoch 7/11 30/30 [==============================] - 3s 99ms/step - loss: 1.1663 - accuracy: 0.8309 - val_loss: 1.1742 - val_accuracy: 0.8131 Epoch 8/11 30/30 [==============================] - 3s 96ms/step - loss: 0.9521 - accuracy: 0.8855 - val_loss: 1.0665 - val_accuracy: 0.8322 Epoch 9/11 30/30 [==============================] - 3s 98ms/step - loss: 0.8334 - accuracy: 0.9136 - val_loss: 1.0044 - val_accuracy: 0.8417 Epoch 10/11 30/30 [==============================] - 3s 90ms/step - loss: 0.7526 - accuracy: 0.9314 - val_loss: 0.9604 - val_accuracy: 0.8492 Epoch 11/11 30/30 [==============================] - 3s 94ms/step - loss: 0.6902 - accuracy: 0.9435 - val_loss: 0.9220 - val_accuracy: 0.8518 Trial 69 Complete [00h 00m 21s] val_accuracy: 0.8518322110176086 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 17m 01s Search: Running Trial #70 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 64 |512 |width none |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.1 |0.1 |dropout 512 |32 |batch_size rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 2 |2 |tuner/bracket 1 |1 |tuner/round 0059 |0056 |tuner/trial_id Epoch 5/11 30/30 [==============================] - 2s 41ms/step - loss: 2.8982 - accuracy: 0.3168 - val_loss: 2.8277 - val_accuracy: 0.4445 Epoch 6/11 30/30 [==============================] - 1s 37ms/step - loss: 2.4695 - accuracy: 0.5856 - val_loss: 2.1271 - val_accuracy: 0.6431 Epoch 7/11 30/30 [==============================] - 1s 37ms/step - loss: 1.9425 - accuracy: 0.7048 - val_loss: 1.9446 - val_accuracy: 0.6861 Epoch 8/11 30/30 [==============================] - 1s 37ms/step - loss: 1.4934 - accuracy: 0.7668 - val_loss: 1.2190 - val_accuracy: 0.7743 Epoch 9/11 30/30 [==============================] - 1s 36ms/step - loss: 0.9171 - accuracy: 0.8466 - val_loss: 0.9137 - val_accuracy: 0.8232 Epoch 10/11 30/30 [==============================] - 1s 37ms/step - loss: 0.6957 - accuracy: 0.8862 - val_loss: 0.8203 - val_accuracy: 0.8279 Epoch 11/11 30/30 [==============================] - 1s 35ms/step - loss: 0.6353 - accuracy: 0.8971 - val_loss: 0.8077 - val_accuracy: 0.8279 Trial 70 Complete [00h 00m 09s] val_accuracy: 0.8279341459274292 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 17m 09s Search: Running Trial #71 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |512 |width l2 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.4 |0.1 |dropout 512 |32 |batch_size rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 2 |2 |tuner/bracket 1 |1 |tuner/round 0063 |0056 |tuner/trial_id Epoch 5/11 30/30 [==============================] - 6s 176ms/step - loss: 3.0256 - accuracy: 0.2950 - val_loss: 2.9777 - val_accuracy: 0.4742 Epoch 6/11 30/30 [==============================] - 5s 167ms/step - loss: 2.9186 - accuracy: 0.4923 - val_loss: 2.8505 - val_accuracy: 0.4663 Epoch 7/11 30/30 [==============================] - 5s 184ms/step - loss: 2.7635 - accuracy: 0.5214 - val_loss: 2.6777 - val_accuracy: 0.5204 Epoch 8/11 30/30 [==============================] - 7s 233ms/step - loss: 2.5771 - accuracy: 0.5785 - val_loss: 2.4975 - val_accuracy: 0.5640 Epoch 9/11 30/30 [==============================] - 7s 237ms/step - loss: 2.3944 - accuracy: 0.6074 - val_loss: 2.3307 - val_accuracy: 0.6012 Epoch 10/11 30/30 [==============================] - 7s 227ms/step - loss: 2.2266 - accuracy: 0.6503 - val_loss: 2.1824 - val_accuracy: 0.6394 Epoch 11/11 30/30 [==============================] - 7s 234ms/step - loss: 2.0765 - accuracy: 0.6803 - val_loss: 2.0515 - val_accuracy: 0.6633 Trial 71 Complete [00h 00m 44s] val_accuracy: 0.6633032560348511 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 17m 54s Search: Running Trial #72 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 256 |512 |width l2 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 256 |32 |batch_size rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 11 |11 |tuner/epochs 4 |4 |tuner/initial_epoch 2 |2 |tuner/bracket 1 |1 |tuner/round 0053 |0056 |tuner/trial_id Epoch 5/11 59/59 [==============================] - 4s 61ms/step - loss: 2.9317 - accuracy: 0.1723 - val_loss: 2.8961 - val_accuracy: 0.2469 Epoch 6/11 59/59 [==============================] - 4s 62ms/step - loss: 2.8102 - accuracy: 0.3706 - val_loss: 2.7328 - val_accuracy: 0.4588 Epoch 7/11 59/59 [==============================] - 4s 64ms/step - loss: 2.6995 - accuracy: 0.4859 - val_loss: 2.6924 - val_accuracy: 0.4918 Epoch 8/11 59/59 [==============================] - 4s 61ms/step - loss: 2.5787 - accuracy: 0.5263 - val_loss: 2.4667 - val_accuracy: 0.5497 Epoch 9/11 59/59 [==============================] - 4s 62ms/step - loss: 2.3620 - accuracy: 0.5927 - val_loss: 2.2880 - val_accuracy: 0.6065 Epoch 10/11 59/59 [==============================] - 3s 57ms/step - loss: 2.2224 - accuracy: 0.6363 - val_loss: 2.2068 - val_accuracy: 0.6245 Epoch 11/11 59/59 [==============================] - 4s 60ms/step - loss: 2.1751 - accuracy: 0.6537 - val_loss: 2.1935 - val_accuracy: 0.6277 Trial 72 Complete [00h 00m 26s] val_accuracy: 0.6277217268943787 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 18m 19s Search: Running Trial #73 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |512 |width l1_l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.1 |0.1 |dropout 32 |32 |batch_size sgd |sgd |optimizer 0.01 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 32 |11 |tuner/epochs 11 |4 |tuner/initial_epoch 2 |2 |tuner/bracket 2 |1 |tuner/round 0067 |0056 |tuner/trial_id Epoch 12/32 471/471 [==============================] - 7s 13ms/step - loss: 1.0436 - accuracy: 0.7238 - val_loss: 0.4406 - val_accuracy: 0.8800 Epoch 13/32 471/471 [==============================] - 6s 13ms/step - loss: 0.1904 - accuracy: 0.9551 - val_loss: 0.3776 - val_accuracy: 0.8996 Epoch 14/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0592 - accuracy: 0.9897 - val_loss: 0.3793 - val_accuracy: 0.9033 Epoch 15/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0243 - accuracy: 0.9958 - val_loss: 0.3853 - val_accuracy: 0.8991 Epoch 16/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0151 - accuracy: 0.9983 - val_loss: 0.3982 - val_accuracy: 0.9007 Epoch 17/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0116 - accuracy: 0.9981 - val_loss: 0.3967 - val_accuracy: 0.9023 Epoch 18/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0081 - accuracy: 0.9989 - val_loss: 0.4304 - val_accuracy: 0.8991 Epoch 19/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0084 - accuracy: 0.9989 - val_loss: 0.4359 - val_accuracy: 0.9023 Epoch 20/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0060 - accuracy: 0.9992 - val_loss: 0.4267 - val_accuracy: 0.9033 Epoch 21/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0060 - accuracy: 0.9993 - val_loss: 0.4309 - val_accuracy: 0.9055 Epoch 22/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0081 - accuracy: 0.9989 - val_loss: 0.4589 - val_accuracy: 0.9023 Epoch 23/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0065 - accuracy: 0.9990 - val_loss: 0.4648 - val_accuracy: 0.9044 Epoch 24/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0073 - accuracy: 0.9985 - val_loss: 0.4822 - val_accuracy: 0.9007 Epoch 25/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0060 - accuracy: 0.9987 - val_loss: 0.4872 - val_accuracy: 0.9055 Epoch 26/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0044 - accuracy: 0.9993 - val_loss: 0.4941 - val_accuracy: 0.9007 Epoch 27/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0041 - accuracy: 0.9992 - val_loss: 0.5066 - val_accuracy: 0.8959 Epoch 28/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0045 - accuracy: 0.9991 - val_loss: 0.4949 - val_accuracy: 0.9018 Epoch 29/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0030 - accuracy: 0.9993 - val_loss: 0.5196 - val_accuracy: 0.8991 Epoch 30/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0035 - accuracy: 0.9992 - val_loss: 0.5132 - val_accuracy: 0.9002 Epoch 31/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0024 - accuracy: 0.9993 - val_loss: 0.5277 - val_accuracy: 0.9018 Epoch 32/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0033 - accuracy: 0.9991 - val_loss: 0.5261 - val_accuracy: 0.9018 Trial 73 Complete [00h 02m 08s] val_accuracy: 0.9054700136184692 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 20m 27s Search: Running Trial #74 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 256 |512 |width l2 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 512 |32 |batch_size rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate none |exponential |learning_rate_schedule 32 |11 |tuner/epochs 11 |4 |tuner/initial_epoch 2 |2 |tuner/bracket 2 |1 |tuner/round 0068 |0056 |tuner/trial_id Epoch 12/32 30/30 [==============================] - 3s 92ms/step - loss: 2.6542 - accuracy: 0.4296 - val_loss: 2.0922 - val_accuracy: 0.6373 Epoch 13/32 30/30 [==============================] - 3s 97ms/step - loss: 1.6659 - accuracy: 0.7218 - val_loss: 1.4025 - val_accuracy: 0.7552 Epoch 14/32 30/30 [==============================] - 3s 105ms/step - loss: 1.1677 - accuracy: 0.8282 - val_loss: 1.1710 - val_accuracy: 0.8062 Epoch 15/32 30/30 [==============================] - 3s 95ms/step - loss: 0.9552 - accuracy: 0.8850 - val_loss: 1.0687 - val_accuracy: 0.8269 Epoch 16/32 30/30 [==============================] - 3s 98ms/step - loss: 0.8347 - accuracy: 0.9128 - val_loss: 1.0025 - val_accuracy: 0.8471 Epoch 17/32 30/30 [==============================] - 4s 126ms/step - loss: 0.7529 - accuracy: 0.9315 - val_loss: 0.9561 - val_accuracy: 0.8524 Epoch 18/32 30/30 [==============================] - 3s 98ms/step - loss: 0.6939 - accuracy: 0.9424 - val_loss: 0.9308 - val_accuracy: 0.8555 Epoch 19/32 30/30 [==============================] - 3s 84ms/step - loss: 0.6450 - accuracy: 0.9501 - val_loss: 0.9037 - val_accuracy: 0.8550 Epoch 20/32 30/30 [==============================] - 3s 85ms/step - loss: 0.6078 - accuracy: 0.9593 - val_loss: 0.8939 - val_accuracy: 0.8545 Epoch 21/32 30/30 [==============================] - 3s 97ms/step - loss: 0.5777 - accuracy: 0.9602 - val_loss: 0.8736 - val_accuracy: 0.8593 Epoch 22/32 30/30 [==============================] - 3s 84ms/step - loss: 0.5470 - accuracy: 0.9656 - val_loss: 0.8507 - val_accuracy: 0.8593 Epoch 23/32 30/30 [==============================] - 3s 97ms/step - loss: 0.5281 - accuracy: 0.9679 - val_loss: 0.8441 - val_accuracy: 0.8625 Epoch 24/32 30/30 [==============================] - 3s 84ms/step - loss: 0.5060 - accuracy: 0.9718 - val_loss: 0.8275 - val_accuracy: 0.8587 Epoch 25/32 30/30 [==============================] - 3s 94ms/step - loss: 0.4853 - accuracy: 0.9736 - val_loss: 0.8164 - val_accuracy: 0.8651 Epoch 26/32 30/30 [==============================] - 3s 98ms/step - loss: 0.4688 - accuracy: 0.9742 - val_loss: 0.8004 - val_accuracy: 0.8683 Epoch 27/32 30/30 [==============================] - 3s 84ms/step - loss: 0.4530 - accuracy: 0.9768 - val_loss: 0.7979 - val_accuracy: 0.8625 Epoch 28/32 30/30 [==============================] - 3s 102ms/step - loss: 0.4398 - accuracy: 0.9790 - val_loss: 0.7793 - val_accuracy: 0.8699 Epoch 29/32 30/30 [==============================] - 3s 85ms/step - loss: 0.4307 - accuracy: 0.9776 - val_loss: 0.7778 - val_accuracy: 0.8625 Epoch 30/32 30/30 [==============================] - 3s 85ms/step - loss: 0.4160 - accuracy: 0.9803 - val_loss: 0.7741 - val_accuracy: 0.8672 Epoch 31/32 30/30 [==============================] - 3s 84ms/step - loss: 0.4102 - accuracy: 0.9805 - val_loss: 0.7702 - val_accuracy: 0.8672 Epoch 32/32 30/30 [==============================] - 3s 85ms/step - loss: 0.3962 - accuracy: 0.9816 - val_loss: 0.7713 - val_accuracy: 0.8571 Trial 74 Complete [00h 00m 59s] val_accuracy: 0.8698884844779968 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 21m 26s Search: Running Trial #75 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 256 |512 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.2 |0.1 |dropout 256 |32 |batch_size adam |sgd |optimizer 0.001 |0.01 |learning_rate none |exponential |learning_rate_schedule 11 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/11 59/59 [==============================] - 2s 30ms/step - loss: 2.9374 - accuracy: 0.1477 - val_loss: 2.8669 - val_accuracy: 0.2284 Epoch 2/11 59/59 [==============================] - 2s 32ms/step - loss: 2.8149 - accuracy: 0.2982 - val_loss: 2.7562 - val_accuracy: 0.3356 Epoch 3/11 59/59 [==============================] - 2s 28ms/step - loss: 2.7097 - accuracy: 0.3882 - val_loss: 2.6558 - val_accuracy: 0.3962 Epoch 4/11 59/59 [==============================] - 2s 30ms/step - loss: 2.6144 - accuracy: 0.4327 - val_loss: 2.5667 - val_accuracy: 0.4307 Epoch 5/11 59/59 [==============================] - 2s 28ms/step - loss: 2.5293 - accuracy: 0.4675 - val_loss: 2.4848 - val_accuracy: 0.4732 Epoch 6/11 59/59 [==============================] - 2s 28ms/step - loss: 2.4508 - accuracy: 0.4896 - val_loss: 2.4078 - val_accuracy: 0.4934 Epoch 7/11 59/59 [==============================] - 2s 28ms/step - loss: 2.3785 - accuracy: 0.5107 - val_loss: 2.3383 - val_accuracy: 0.5284 Epoch 8/11 59/59 [==============================] - 2s 28ms/step - loss: 2.3119 - accuracy: 0.5245 - val_loss: 2.2762 - val_accuracy: 0.5353 Epoch 9/11 59/59 [==============================] - 2s 28ms/step - loss: 2.2508 - accuracy: 0.5372 - val_loss: 2.2189 - val_accuracy: 0.5385 Epoch 10/11 59/59 [==============================] - 2s 28ms/step - loss: 2.1938 - accuracy: 0.5451 - val_loss: 2.1626 - val_accuracy: 0.5528 Epoch 11/11 59/59 [==============================] - 2s 28ms/step - loss: 2.1409 - accuracy: 0.5541 - val_loss: 2.1138 - val_accuracy: 0.5666 Trial 75 Complete [00h 00m 19s] val_accuracy: 0.5666489601135254 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 21m 46s Search: Running Trial #76 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 256 |512 |width none |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 512 |32 |batch_size sgd |sgd |optimizer 0.001 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 11 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/11 30/30 [==============================] - 3s 95ms/step - loss: 2.9832 - accuracy: 0.2506 - val_loss: 2.9760 - val_accuracy: 0.3797 Epoch 2/11 30/30 [==============================] - 3s 98ms/step - loss: 2.9509 - accuracy: 0.4891 - val_loss: 2.9259 - val_accuracy: 0.5231 Epoch 3/11 30/30 [==============================] - 3s 97ms/step - loss: 2.9098 - accuracy: 0.5826 - val_loss: 2.9090 - val_accuracy: 0.5443 Epoch 4/11 30/30 [==============================] - 3s 94ms/step - loss: 2.8512 - accuracy: 0.6077 - val_loss: 2.7900 - val_accuracy: 0.5900 Epoch 5/11 30/30 [==============================] - 4s 120ms/step - loss: 2.7179 - accuracy: 0.6164 - val_loss: 2.6665 - val_accuracy: 0.5937 Epoch 6/11 30/30 [==============================] - 3s 102ms/step - loss: 2.6150 - accuracy: 0.6378 - val_loss: 2.6035 - val_accuracy: 0.6102 Epoch 7/11 30/30 [==============================] - 3s 88ms/step - loss: 2.5781 - accuracy: 0.6456 - val_loss: 2.5934 - val_accuracy: 0.6102 Epoch 8/11 30/30 [==============================] - 4s 119ms/step - loss: 2.4837 - accuracy: 0.6473 - val_loss: 2.4028 - val_accuracy: 0.6288 Epoch 9/11 30/30 [==============================] - 3s 98ms/step - loss: 2.2816 - accuracy: 0.6816 - val_loss: 2.2149 - val_accuracy: 0.6665 Epoch 10/11 30/30 [==============================] - 3s 101ms/step - loss: 2.0918 - accuracy: 0.7138 - val_loss: 2.0520 - val_accuracy: 0.6920 Epoch 11/11 30/30 [==============================] - 3s 108ms/step - loss: 1.9365 - accuracy: 0.7344 - val_loss: 1.9294 - val_accuracy: 0.7074 Trial 76 Complete [00h 00m 34s] val_accuracy: 0.7073818445205688 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 22m 20s Search: Running Trial #77 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 64 |512 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.4 |0.1 |dropout 512 |32 |batch_size adam |sgd |optimizer 0.001 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 11 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/11 5/30 [====>.........................] - ETA: 0s - loss: 5.3072 - accuracy: 0.0504WARNING:tensorflow:Callback method `on_train_batch_en30/30 [==============================] - 2s 39ms/step - loss: 4.6077 - accuracy: 0.0639 - val_loss: 4.2100 - val_accuracy: 0.0706 Epoch 2/11 30/30 [==============================] - 1s 37ms/step - loss: 3.5734 - accuracy: 0.1119 - val_loss: 3.1422 - val_accuracy: 0.1174 Epoch 3/11 30/30 [==============================] - 1s 33ms/step - loss: 3.0547 - accuracy: 0.1194 - val_loss: 3.0196 - val_accuracy: 0.1126 Epoch 4/11 30/30 [==============================] - 1s 33ms/step - loss: 3.0514 - accuracy: 0.1117 - val_loss: 3.0100 - val_accuracy: 0.1115 Epoch 5/11 30/30 [==============================] - 1s 38ms/step - loss: 2.9983 - accuracy: 0.1187 - val_loss: 2.9903 - val_accuracy: 0.1184 Epoch 6/11 30/30 [==============================] - 1s 33ms/step - loss: 2.9857 - accuracy: 0.1234 - val_loss: 2.9825 - val_accuracy: 0.1158 Epoch 7/11 30/30 [==============================] - 1s 33ms/step - loss: 2.9800 - accuracy: 0.1241 - val_loss: 2.9795 - val_accuracy: 0.1152 Epoch 8/11 30/30 [==============================] - 1s 33ms/step - loss: 3.0061 - accuracy: 0.1259 - val_loss: 2.9841 - val_accuracy: 0.1179 Epoch 9/11 30/30 [==============================] - 1s 34ms/step - loss: 2.9750 - accuracy: 0.1279 - val_loss: 2.9653 - val_accuracy: 0.1232 Epoch 10/11 30/30 [==============================] - 1s 33ms/step - loss: 2.9577 - accuracy: 0.1271 - val_loss: 2.9487 - val_accuracy: 0.1221 Epoch 11/11 30/30 [==============================] - 1s 33ms/step - loss: 2.9420 - accuracy: 0.1281 - val_loss: 2.9340 - val_accuracy: 0.1216 Trial 77 Complete [00h 00m 12s] val_accuracy: 0.12320765107870102 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 22m 32s Search: Running Trial #78 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 64 |512 |width l1_l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 32 |32 |batch_size rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 11 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/11 471/471 [==============================] - 7s 13ms/step - loss: 2.7367 - accuracy: 0.3458 - val_loss: 2.6070 - val_accuracy: 0.4689 Epoch 2/11 471/471 [==============================] - 6s 14ms/step - loss: 2.2085 - accuracy: 0.5598 - val_loss: 1.8910 - val_accuracy: 0.6362 Epoch 3/11 471/471 [==============================] - 6s 14ms/step - loss: 1.7549 - accuracy: 0.6893 - val_loss: 1.7507 - val_accuracy: 0.6792 Epoch 4/11 471/471 [==============================] - 7s 16ms/step - loss: 1.4168 - accuracy: 0.7334 - val_loss: 1.2140 - val_accuracy: 0.7605 Epoch 5/11 471/471 [==============================] - 6s 13ms/step - loss: 0.9991 - accuracy: 0.8164 - val_loss: 0.9786 - val_accuracy: 0.7913 Epoch 6/11 471/471 [==============================] - 6s 13ms/step - loss: 0.8206 - accuracy: 0.8488 - val_loss: 0.8983 - val_accuracy: 0.8040 Epoch 7/11 471/471 [==============================] - 6s 13ms/step - loss: 0.7680 - accuracy: 0.8601 - val_loss: 0.8867 - val_accuracy: 0.8035 Epoch 8/11 471/471 [==============================] - 6s 13ms/step - loss: 0.6742 - accuracy: 0.8692 - val_loss: 0.7110 - val_accuracy: 0.8332 Epoch 9/11 471/471 [==============================] - 6s 13ms/step - loss: 0.5122 - accuracy: 0.8990 - val_loss: 0.6129 - val_accuracy: 0.8433 Epoch 10/11 471/471 [==============================] - 7s 14ms/step - loss: 0.4095 - accuracy: 0.9201 - val_loss: 0.5534 - val_accuracy: 0.8545 Epoch 11/11 471/471 [==============================] - 6s 14ms/step - loss: 0.3430 - accuracy: 0.9359 - val_loss: 0.5151 - val_accuracy: 0.8678 Trial 78 Complete [00h 01m 11s] val_accuracy: 0.8677642345428467 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 23m 44s Search: Running Trial #79 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 512 |512 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.2 |0.1 |dropout 512 |32 |batch_size sgd |sgd |optimizer 0.001 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 11 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/11 30/30 [==============================] - 6s 177ms/step - loss: 4.3520 - accuracy: 0.1491 - val_loss: 4.0495 - val_accuracy: 0.1721 Epoch 2/11 30/30 [==============================] - 6s 188ms/step - loss: 3.5138 - accuracy: 0.1960 - val_loss: 3.1293 - val_accuracy: 0.1992 Epoch 3/11 30/30 [==============================] - 5s 170ms/step - loss: 3.0391 - accuracy: 0.1936 - val_loss: 3.0026 - val_accuracy: 0.1806 Epoch 4/11 30/30 [==============================] - 7s 231ms/step - loss: 2.9741 - accuracy: 0.2016 - val_loss: 2.8984 - val_accuracy: 0.2055 Epoch 5/11 30/30 [==============================] - 7s 227ms/step - loss: 2.8584 - accuracy: 0.2124 - val_loss: 2.8189 - val_accuracy: 0.2167 Epoch 6/11 30/30 [==============================] - 7s 233ms/step - loss: 2.8014 - accuracy: 0.2275 - val_loss: 2.7843 - val_accuracy: 0.2204 Epoch 7/11 30/30 [==============================] - 6s 211ms/step - loss: 2.7810 - accuracy: 0.2364 - val_loss: 2.7774 - val_accuracy: 0.2215 Epoch 8/11 30/30 [==============================] - 7s 227ms/step - loss: 2.7663 - accuracy: 0.2482 - val_loss: 2.7063 - val_accuracy: 0.2767 Epoch 9/11 30/30 [==============================] - 7s 230ms/step - loss: 2.6683 - accuracy: 0.3000 - val_loss: 2.6242 - val_accuracy: 0.3064 Epoch 10/11 30/30 [==============================] - 7s 231ms/step - loss: 2.5910 - accuracy: 0.3326 - val_loss: 2.5556 - val_accuracy: 0.3383 Epoch 11/11 30/30 [==============================] - 7s 236ms/step - loss: 2.5283 - accuracy: 0.3539 - val_loss: 2.5033 - val_accuracy: 0.3537 Trial 79 Complete [00h 01m 11s] val_accuracy: 0.3536909222602844 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 24m 55s Search: Running Trial #80 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 64 |512 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.5 |0.1 |dropout 128 |32 |batch_size rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 11 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/11 118/118 [==============================] - 4s 34ms/step - loss: 3.0150 - accuracy: 0.0566 - val_loss: 3.0069 - val_accuracy: 0.0531 Epoch 2/11 118/118 [==============================] - 4s 35ms/step - loss: 2.9840 - accuracy: 0.0694 - val_loss: 2.9688 - val_accuracy: 0.0674 Epoch 3/11 118/118 [==============================] - 5s 44ms/step - loss: 2.9621 - accuracy: 0.0898 - val_loss: 2.9615 - val_accuracy: 0.0738 Epoch 4/11 118/118 [==============================] - 4s 34ms/step - loss: 2.9434 - accuracy: 0.1251 - val_loss: 2.9282 - val_accuracy: 0.1625 Epoch 5/11 118/118 [==============================] - 4s 34ms/step - loss: 2.9143 - accuracy: 0.1971 - val_loss: 2.9061 - val_accuracy: 0.2135 Epoch 6/11 118/118 [==============================] - 4s 33ms/step - loss: 2.8986 - accuracy: 0.2254 - val_loss: 2.8961 - val_accuracy: 0.2358 Epoch 7/11 118/118 [==============================] - 4s 34ms/step - loss: 2.8928 - accuracy: 0.2393 - val_loss: 2.8944 - val_accuracy: 0.2443 Epoch 8/11 118/118 [==============================] - 4s 33ms/step - loss: 2.8774 - accuracy: 0.2793 - val_loss: 2.8632 - val_accuracy: 0.3298 Epoch 9/11 118/118 [==============================] - 4s 34ms/step - loss: 2.8466 - accuracy: 0.3467 - val_loss: 2.8342 - val_accuracy: 0.3797 Epoch 10/11 118/118 [==============================] - 4s 34ms/step - loss: 2.8191 - accuracy: 0.3856 - val_loss: 2.8093 - val_accuracy: 0.4047 Epoch 11/11 118/118 [==============================] - 4s 33ms/step - loss: 2.7951 - accuracy: 0.4130 - val_loss: 2.7897 - val_accuracy: 0.4180 Trial 80 Complete [00h 00m 46s] val_accuracy: 0.4179500937461853 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 25m 41s Search: Running Trial #81 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 512 |512 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 1024 |32 |batch_size rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 11 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/11 15/15 [==============================] - 2s 116ms/step - loss: 2.8404 - accuracy: 0.2138 - val_loss: 2.6495 - val_accuracy: 0.3160 Epoch 2/11 15/15 [==============================] - 2s 121ms/step - loss: 2.5384 - accuracy: 0.4049 - val_loss: 2.3968 - val_accuracy: 0.4859 Epoch 3/11 15/15 [==============================] - 2s 118ms/step - loss: 2.3111 - accuracy: 0.4971 - val_loss: 2.2054 - val_accuracy: 0.5449 Epoch 4/11 15/15 [==============================] - 2s 107ms/step - loss: 2.1409 - accuracy: 0.5408 - val_loss: 2.0575 - val_accuracy: 0.5651 Epoch 5/11 15/15 [==============================] - 2s 109ms/step - loss: 2.0095 - accuracy: 0.5649 - val_loss: 1.9437 - val_accuracy: 0.5698 Epoch 6/11 15/15 [==============================] - 2s 107ms/step - loss: 1.9007 - accuracy: 0.5871 - val_loss: 1.8464 - val_accuracy: 0.6070 Epoch 7/11 15/15 [==============================] - 2s 108ms/step - loss: 1.8151 - accuracy: 0.6017 - val_loss: 1.7710 - val_accuracy: 0.6129 Epoch 8/11 15/15 [==============================] - 2s 107ms/step - loss: 1.7426 - accuracy: 0.6144 - val_loss: 1.7041 - val_accuracy: 0.6208 Epoch 9/11 15/15 [==============================] - 2s 109ms/step - loss: 1.6831 - accuracy: 0.6259 - val_loss: 1.6521 - val_accuracy: 0.6277 Epoch 10/11 15/15 [==============================] - 2s 112ms/step - loss: 1.6328 - accuracy: 0.6274 - val_loss: 1.6039 - val_accuracy: 0.6357 Epoch 11/11 15/15 [==============================] - 2s 104ms/step - loss: 1.5888 - accuracy: 0.6388 - val_loss: 1.5624 - val_accuracy: 0.6320 Trial 81 Complete [00h 00m 19s] val_accuracy: 0.6356877088546753 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 25m 59s Search: Running Trial #82 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 64 |512 |width none |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 64 |32 |batch_size rmsprop |sgd |optimizer 0.0001 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 11 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/11 236/236 [==============================] - 5s 20ms/step - loss: 2.9844 - accuracy: 0.0877 - val_loss: 2.9528 - val_accuracy: 0.0961 Epoch 2/11 236/236 [==============================] - 5s 21ms/step - loss: 2.9256 - accuracy: 0.1232 - val_loss: 2.9029 - val_accuracy: 0.1737 Epoch 3/11 236/236 [==============================] - 5s 20ms/step - loss: 2.8800 - accuracy: 0.2139 - val_loss: 2.8596 - val_accuracy: 0.2560 Epoch 4/11 236/236 [==============================] - 5s 20ms/step - loss: 2.8368 - accuracy: 0.2925 - val_loss: 2.8177 - val_accuracy: 0.3144 Epoch 5/11 236/236 [==============================] - 5s 20ms/step - loss: 2.7947 - accuracy: 0.3585 - val_loss: 2.7771 - val_accuracy: 0.3707 Epoch 6/11 236/236 [==============================] - 5s 20ms/step - loss: 2.7545 - accuracy: 0.4096 - val_loss: 2.7374 - val_accuracy: 0.4068 Epoch 7/11 236/236 [==============================] - 5s 20ms/step - loss: 2.7138 - accuracy: 0.4412 - val_loss: 2.6985 - val_accuracy: 0.4434 Epoch 8/11 236/236 [==============================] - 5s 19ms/step - loss: 2.6745 - accuracy: 0.4676 - val_loss: 2.6607 - val_accuracy: 0.4572 Epoch 9/11 236/236 [==============================] - 5s 20ms/step - loss: 2.6370 - accuracy: 0.4894 - val_loss: 2.6238 - val_accuracy: 0.4796 Epoch 10/11 236/236 [==============================] - 5s 21ms/step - loss: 2.5993 - accuracy: 0.5042 - val_loss: 2.5884 - val_accuracy: 0.4965 Epoch 11/11 236/236 [==============================] - 5s 20ms/step - loss: 2.5636 - accuracy: 0.5270 - val_loss: 2.5543 - val_accuracy: 0.5130 Trial 82 Complete [00h 00m 53s] val_accuracy: 0.5130111575126648 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 26m 53s Search: Running Trial #83 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 64 |512 |width l1_l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 32 |32 |batch_size rmsprop |sgd |optimizer 0.001 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 32 |11 |tuner/epochs 11 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 1 |1 |tuner/round 0077 |0056 |tuner/trial_id Epoch 12/32 471/471 [==============================] - 7s 13ms/step - loss: 2.7353 - accuracy: 0.3453 - val_loss: 2.6080 - val_accuracy: 0.4663 Epoch 13/32 471/471 [==============================] - 6s 14ms/step - loss: 2.2081 - accuracy: 0.5767 - val_loss: 1.8889 - val_accuracy: 0.6532 Epoch 14/32 471/471 [==============================] - 6s 14ms/step - loss: 1.7526 - accuracy: 0.6912 - val_loss: 1.7475 - val_accuracy: 0.6856 Epoch 15/32 471/471 [==============================] - 6s 13ms/step - loss: 1.4132 - accuracy: 0.7397 - val_loss: 1.2093 - val_accuracy: 0.7616 Epoch 16/32 471/471 [==============================] - 6s 13ms/step - loss: 0.9981 - accuracy: 0.8142 - val_loss: 0.9776 - val_accuracy: 0.7918 Epoch 17/32 471/471 [==============================] - 6s 13ms/step - loss: 0.8221 - accuracy: 0.8485 - val_loss: 0.8972 - val_accuracy: 0.8008 Epoch 18/32 471/471 [==============================] - 6s 13ms/step - loss: 0.7693 - accuracy: 0.8587 - val_loss: 0.8851 - val_accuracy: 0.8019 Epoch 19/32 471/471 [==============================] - 6s 13ms/step - loss: 0.6779 - accuracy: 0.8648 - val_loss: 0.7116 - val_accuracy: 0.8317 Epoch 20/32 471/471 [==============================] - 6s 13ms/step - loss: 0.5150 - accuracy: 0.8974 - val_loss: 0.6109 - val_accuracy: 0.8471 Epoch 21/32 471/471 [==============================] - 6s 13ms/step - loss: 0.4118 - accuracy: 0.9201 - val_loss: 0.5517 - val_accuracy: 0.8609 Epoch 22/32 471/471 [==============================] - 6s 13ms/step - loss: 0.3450 - accuracy: 0.9355 - val_loss: 0.5167 - val_accuracy: 0.8662 Epoch 23/32 471/471 [==============================] - 6s 13ms/step - loss: 0.3040 - accuracy: 0.9452 - val_loss: 0.4927 - val_accuracy: 0.8694 Epoch 24/32 471/471 [==============================] - 6s 13ms/step - loss: 0.2789 - accuracy: 0.9511 - val_loss: 0.4817 - val_accuracy: 0.8763 Epoch 25/32 471/471 [==============================] - 6s 13ms/step - loss: 0.2657 - accuracy: 0.9537 - val_loss: 0.4773 - val_accuracy: 0.8747 Epoch 26/32 471/471 [==============================] - 6s 13ms/step - loss: 0.2613 - accuracy: 0.9558 - val_loss: 0.4766 - val_accuracy: 0.8752 Epoch 27/32 471/471 [==============================] - 6s 14ms/step - loss: 0.2479 - accuracy: 0.9555 - val_loss: 0.4359 - val_accuracy: 0.8853 Epoch 28/32 471/471 [==============================] - 6s 13ms/step - loss: 0.1962 - accuracy: 0.9657 - val_loss: 0.4109 - val_accuracy: 0.8927 Epoch 29/32 471/471 [==============================] - 6s 13ms/step - loss: 0.1571 - accuracy: 0.9753 - val_loss: 0.3926 - val_accuracy: 0.8943 Epoch 30/32 471/471 [==============================] - 6s 13ms/step - loss: 0.1288 - accuracy: 0.9818 - val_loss: 0.3780 - val_accuracy: 0.8996 Epoch 31/32 471/471 [==============================] - 6s 13ms/step - loss: 0.1051 - accuracy: 0.9861 - val_loss: 0.3685 - val_accuracy: 0.9018 Epoch 32/32 471/471 [==============================] - 6s 13ms/step - loss: 0.0885 - accuracy: 0.9889 - val_loss: 0.3598 - val_accuracy: 0.9023 Trial 83 Complete [00h 02m 13s] val_accuracy: 0.9022836089134216 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 29m 05s Search: Running Trial #84 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 256 |512 |width none |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 512 |32 |batch_size sgd |sgd |optimizer 0.001 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 32 |11 |tuner/epochs 11 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 1 |1 |tuner/round 0075 |0056 |tuner/trial_id Epoch 12/32 30/30 [==============================] - 3s 94ms/step - loss: 2.9826 - accuracy: 0.2365 - val_loss: 2.9753 - val_accuracy: 0.3633 Epoch 13/32 30/30 [==============================] - 3s 100ms/step - loss: 2.9490 - accuracy: 0.5033 - val_loss: 2.9230 - val_accuracy: 0.5412 Epoch 14/32 30/30 [==============================] - 3s 88ms/step - loss: 2.9062 - accuracy: 0.5772 - val_loss: 2.9052 - val_accuracy: 0.5401 Epoch 15/32 30/30 [==============================] - 3s 98ms/step - loss: 2.8457 - accuracy: 0.5896 - val_loss: 2.7832 - val_accuracy: 0.5603 Epoch 16/32 30/30 [==============================] - 3s 94ms/step - loss: 2.7110 - accuracy: 0.6026 - val_loss: 2.6596 - val_accuracy: 0.5709 Epoch 17/32 30/30 [==============================] - 3s 95ms/step - loss: 2.6097 - accuracy: 0.6135 - val_loss: 2.5980 - val_accuracy: 0.5767 Epoch 18/32 30/30 [==============================] - 3s 97ms/step - loss: 2.5733 - accuracy: 0.6212 - val_loss: 2.5882 - val_accuracy: 0.5789 Epoch 19/32 30/30 [==============================] - 3s 99ms/step - loss: 2.4817 - accuracy: 0.6282 - val_loss: 2.4023 - val_accuracy: 0.6091 Epoch 20/32 30/30 [==============================] - 3s 95ms/step - loss: 2.2855 - accuracy: 0.6678 - val_loss: 2.2205 - val_accuracy: 0.6559 Epoch 21/32 30/30 [==============================] - 3s 94ms/step - loss: 2.1011 - accuracy: 0.7018 - val_loss: 2.0617 - val_accuracy: 0.6782 Epoch 22/32 30/30 [==============================] - 3s 93ms/step - loss: 1.9496 - accuracy: 0.7281 - val_loss: 1.9415 - val_accuracy: 0.7031 Epoch 23/32 30/30 [==============================] - 3s 98ms/step - loss: 1.8400 - accuracy: 0.7467 - val_loss: 1.8633 - val_accuracy: 0.7138 Epoch 24/32 30/30 [==============================] - 3s 98ms/step - loss: 1.7740 - accuracy: 0.7542 - val_loss: 1.8215 - val_accuracy: 0.7201 Epoch 25/32 30/30 [==============================] - 3s 97ms/step - loss: 1.7422 - accuracy: 0.7598 - val_loss: 1.8056 - val_accuracy: 0.7207 Epoch 26/32 30/30 [==============================] - 3s 92ms/step - loss: 1.7333 - accuracy: 0.7598 - val_loss: 1.8033 - val_accuracy: 0.7212 Epoch 27/32 30/30 [==============================] - 3s 94ms/step - loss: 1.6514 - accuracy: 0.7686 - val_loss: 1.6384 - val_accuracy: 0.7398 Epoch 28/32 30/30 [==============================] - 3s 98ms/step - loss: 1.4790 - accuracy: 0.7895 - val_loss: 1.4892 - val_accuracy: 0.7562 Epoch 29/32 30/30 [==============================] - 3s 116ms/step - loss: 1.3258 - accuracy: 0.8116 - val_loss: 1.3603 - val_accuracy: 0.7663 Epoch 30/32 30/30 [==============================] - 3s 98ms/step - loss: 1.1942 - accuracy: 0.8264 - val_loss: 1.2530 - val_accuracy: 0.7796 Epoch 31/32 30/30 [==============================] - 3s 91ms/step - loss: 1.0857 - accuracy: 0.8398 - val_loss: 1.1650 - val_accuracy: 0.7939 Epoch 32/32 30/30 [==============================] - 3s 97ms/step - loss: 0.9968 - accuracy: 0.8535 - val_loss: 1.0951 - val_accuracy: 0.8030 Trial 84 Complete [00h 01m 01s] val_accuracy: 0.8029739856719971 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 30m 07s Search: Running Trial #85 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 512 |512 |width l1 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 1024 |32 |batch_size rmsprop |sgd |optimizer 0.01 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 32 |11 |tuner/epochs 11 |4 |tuner/initial_epoch 1 |2 |tuner/bracket 1 |1 |tuner/round 0080 |0056 |tuner/trial_id Epoch 12/32 15/15 [==============================] - 2s 116ms/step - loss: 2.8405 - accuracy: 0.2159 - val_loss: 2.6486 - val_accuracy: 0.3696 Epoch 13/32 15/15 [==============================] - 2s 114ms/step - loss: 2.5293 - accuracy: 0.4140 - val_loss: 2.3892 - val_accuracy: 0.4488 Epoch 14/32 15/15 [==============================] - 2s 108ms/step - loss: 2.3041 - accuracy: 0.4931 - val_loss: 2.1965 - val_accuracy: 0.5258 Epoch 15/32 15/15 [==============================] - 2s 113ms/step - loss: 2.1333 - accuracy: 0.5343 - val_loss: 2.0500 - val_accuracy: 0.5619 Epoch 16/32 15/15 [==============================] - 2s 112ms/step - loss: 2.0006 - accuracy: 0.5635 - val_loss: 1.9318 - val_accuracy: 0.5767 Epoch 17/32 15/15 [==============================] - 2s 106ms/step - loss: 1.8926 - accuracy: 0.5887 - val_loss: 1.8383 - val_accuracy: 0.6144 Epoch 18/32 15/15 [==============================] - 2s 104ms/step - loss: 1.8080 - accuracy: 0.6056 - val_loss: 1.7652 - val_accuracy: 0.6139 Epoch 19/32 15/15 [==============================] - 2s 111ms/step - loss: 1.7357 - accuracy: 0.6150 - val_loss: 1.6983 - val_accuracy: 0.6219 Epoch 20/32 15/15 [==============================] - 2s 111ms/step - loss: 1.6772 - accuracy: 0.6237 - val_loss: 1.6449 - val_accuracy: 0.6298 Epoch 21/32 15/15 [==============================] - 2s 107ms/step - loss: 1.6270 - accuracy: 0.6306 - val_loss: 1.6001 - val_accuracy: 0.6357 Epoch 22/32 15/15 [==============================] - 2s 104ms/step - loss: 1.5847 - accuracy: 0.6356 - val_loss: 1.5625 - val_accuracy: 0.6336 Epoch 23/32 15/15 [==============================] - 2s 109ms/step - loss: 1.5473 - accuracy: 0.6409 - val_loss: 1.5254 - val_accuracy: 0.6495 Epoch 24/32 15/15 [==============================] - 2s 104ms/step - loss: 1.5153 - accuracy: 0.6485 - val_loss: 1.4942 - val_accuracy: 0.6389 Epoch 25/32 15/15 [==============================] - 2s 114ms/step - loss: 1.4854 - accuracy: 0.6506 - val_loss: 1.4697 - val_accuracy: 0.6511 Epoch 26/32 15/15 [==============================] - 2s 107ms/step - loss: 1.4588 - accuracy: 0.6558 - val_loss: 1.4444 - val_accuracy: 0.6537 Epoch 27/32 15/15 [==============================] - 2s 110ms/step - loss: 1.4373 - accuracy: 0.6581 - val_loss: 1.4227 - val_accuracy: 0.6543 Epoch 28/32 15/15 [==============================] - 2s 112ms/step - loss: 1.4172 - accuracy: 0.6635 - val_loss: 1.4047 - val_accuracy: 0.6596 Epoch 29/32 15/15 [==============================] - 2s 112ms/step - loss: 1.3969 - accuracy: 0.6643 - val_loss: 1.3854 - val_accuracy: 0.6644 Epoch 30/32 15/15 [==============================] - 2s 107ms/step - loss: 1.3813 - accuracy: 0.6694 - val_loss: 1.3728 - val_accuracy: 0.6670 Epoch 31/32 15/15 [==============================] - 2s 110ms/step - loss: 1.3672 - accuracy: 0.6666 - val_loss: 1.3553 - val_accuracy: 0.6707 Epoch 32/32 15/15 [==============================] - 2s 104ms/step - loss: 1.3537 - accuracy: 0.6722 - val_loss: 1.3435 - val_accuracy: 0.6654 Trial 85 Complete [00h 00m 35s] val_accuracy: 0.670738160610199 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 30m 41s Search: Running Trial #86 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |512 |width l2 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.4 |0.1 |dropout 512 |32 |batch_size sgd |sgd |optimizer 0.001 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 32 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/32 30/30 [==============================] - 4s 114ms/step - loss: 2.9498 - accuracy: 0.0799 - val_loss: 2.9270 - val_accuracy: 0.1126 Epoch 2/32 30/30 [==============================] - 3s 110ms/step - loss: 2.8743 - accuracy: 0.2454 - val_loss: 2.8290 - val_accuracy: 0.3399 Epoch 3/32 30/30 [==============================] - 4s 123ms/step - loss: 2.8087 - accuracy: 0.3702 - val_loss: 2.8066 - val_accuracy: 0.3617 Epoch 4/32 30/30 [==============================] - 3s 115ms/step - loss: 2.7471 - accuracy: 0.4154 - val_loss: 2.6870 - val_accuracy: 0.4477 Epoch 5/32 30/30 [==============================] - 4s 119ms/step - loss: 2.6320 - accuracy: 0.5066 - val_loss: 2.5925 - val_accuracy: 0.5284 Epoch 6/32 30/30 [==============================] - 4s 119ms/step - loss: 2.5570 - accuracy: 0.5372 - val_loss: 2.5468 - val_accuracy: 0.5332 Epoch 7/32 30/30 [==============================] - 3s 111ms/step - loss: 2.5305 - accuracy: 0.5422 - val_loss: 2.5391 - val_accuracy: 0.5353 Epoch 8/32 30/30 [==============================] - 4s 134ms/step - loss: 2.4591 - accuracy: 0.5619 - val_loss: 2.3944 - val_accuracy: 0.5805 Epoch 9/32 30/30 [==============================] - 4s 144ms/step - loss: 2.3111 - accuracy: 0.6024 - val_loss: 2.2551 - val_accuracy: 0.6118 Epoch 10/32 30/30 [==============================] - 3s 117ms/step - loss: 2.1744 - accuracy: 0.6401 - val_loss: 2.1374 - val_accuracy: 0.6415 Epoch 11/32 30/30 [==============================] - 4s 142ms/step - loss: 2.0629 - accuracy: 0.6653 - val_loss: 2.0482 - val_accuracy: 0.6575 Epoch 12/32 30/30 [==============================] - 3s 103ms/step - loss: 1.9825 - accuracy: 0.6737 - val_loss: 1.9873 - val_accuracy: 0.6564 Epoch 13/32 30/30 [==============================] - 4s 118ms/step - loss: 1.9333 - accuracy: 0.6852 - val_loss: 1.9546 - val_accuracy: 0.6649 Epoch 14/32 30/30 [==============================] - 3s 115ms/step - loss: 1.9086 - accuracy: 0.6913 - val_loss: 1.9420 - val_accuracy: 0.6681 Epoch 15/32 30/30 [==============================] - 4s 120ms/step - loss: 1.9011 - accuracy: 0.6908 - val_loss: 1.9401 - val_accuracy: 0.6686 Epoch 16/32 30/30 [==============================] - 4s 126ms/step - loss: 1.8346 - accuracy: 0.7000 - val_loss: 1.8033 - val_accuracy: 0.6867 Epoch 17/32 30/30 [==============================] - 4s 124ms/step - loss: 1.6923 - accuracy: 0.7243 - val_loss: 1.6752 - val_accuracy: 0.7063 Epoch 18/32 30/30 [==============================] - 3s 116ms/step - loss: 1.5618 - accuracy: 0.7438 - val_loss: 1.5605 - val_accuracy: 0.7153 Epoch 19/32 30/30 [==============================] - 3s 106ms/step - loss: 1.4433 - accuracy: 0.7602 - val_loss: 1.4592 - val_accuracy: 0.7408 Epoch 20/32 30/30 [==============================] - 3s 110ms/step - loss: 1.3409 - accuracy: 0.7802 - val_loss: 1.3745 - val_accuracy: 0.7504 Epoch 21/32 30/30 [==============================] - 4s 123ms/step - loss: 1.2543 - accuracy: 0.7940 - val_loss: 1.3036 - val_accuracy: 0.7568 Epoch 22/32 30/30 [==============================] - 3s 109ms/step - loss: 1.1820 - accuracy: 0.8040 - val_loss: 1.2460 - val_accuracy: 0.7663 Epoch 23/32 30/30 [==============================] - 3s 108ms/step - loss: 1.1259 - accuracy: 0.8142 - val_loss: 1.1997 - val_accuracy: 0.7711 Epoch 24/32 30/30 [==============================] - 3s 108ms/step - loss: 1.0789 - accuracy: 0.8201 - val_loss: 1.1642 - val_accuracy: 0.7770 Epoch 25/32 30/30 [==============================] - 4s 145ms/step - loss: 1.0439 - accuracy: 0.8255 - val_loss: 1.1376 - val_accuracy: 0.7791 Epoch 26/32 30/30 [==============================] - 3s 112ms/step - loss: 1.0180 - accuracy: 0.8316 - val_loss: 1.1187 - val_accuracy: 0.7817 Epoch 27/32 30/30 [==============================] - 4s 120ms/step - loss: 1.0004 - accuracy: 0.8322 - val_loss: 1.1056 - val_accuracy: 0.7828 Epoch 28/32 30/30 [==============================] - 3s 108ms/step - loss: 0.9878 - accuracy: 0.8358 - val_loss: 1.0976 - val_accuracy: 0.7839 Epoch 29/32 30/30 [==============================] - 4s 119ms/step - loss: 0.9795 - accuracy: 0.8374 - val_loss: 1.0935 - val_accuracy: 0.7844 Epoch 30/32 30/30 [==============================] - 4s 141ms/step - loss: 0.9756 - accuracy: 0.8370 - val_loss: 1.0920 - val_accuracy: 0.7849 Epoch 31/32 30/30 [==============================] - 3s 102ms/step - loss: 0.9759 - accuracy: 0.8372 - val_loss: 1.0917 - val_accuracy: 0.7849 Epoch 32/32 30/30 [==============================] - 3s 117ms/step - loss: 0.9451 - accuracy: 0.8416 - val_loss: 1.0278 - val_accuracy: 0.7913 Trial 86 Complete [00h 01m 54s] val_accuracy: 0.7912905216217041 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 32m 36s Search: Running Trial #87 Value |Best Value So Far |Hyperparameter uninitialized |initialized |embedding_type 128 |512 |width l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.1 |0.1 |dropout 32 |32 |batch_size sgd |sgd |optimizer 0.01 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 32 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/32 471/471 [==============================] - 4s 7ms/step - loss: 2.3845 - accuracy: 0.4579 - val_loss: 2.0430 - val_accuracy: 0.6314 Epoch 2/32 471/471 [==============================] - 3s 7ms/step - loss: 1.9323 - accuracy: 0.6706 - val_loss: 1.8395 - val_accuracy: 0.7148 Epoch 3/32 471/471 [==============================] - 3s 7ms/step - loss: 1.7330 - accuracy: 0.7385 - val_loss: 1.7305 - val_accuracy: 0.7233 Epoch 4/32 471/471 [==============================] - 3s 7ms/step - loss: 1.6075 - accuracy: 0.7721 - val_loss: 1.6333 - val_accuracy: 0.7552 Epoch 5/32 471/471 [==============================] - 3s 7ms/step - loss: 1.5127 - accuracy: 0.7881 - val_loss: 1.5669 - val_accuracy: 0.7759 Epoch 6/32 471/471 [==============================] - 3s 7ms/step - loss: 1.4218 - accuracy: 0.8106 - val_loss: 1.4973 - val_accuracy: 0.7844 Epoch 7/32 471/471 [==============================] - 3s 7ms/step - loss: 1.3631 - accuracy: 0.8207 - val_loss: 1.4571 - val_accuracy: 0.7918 Epoch 8/32 471/471 [==============================] - 3s 7ms/step - loss: 1.2944 - accuracy: 0.8332 - val_loss: 1.4044 - val_accuracy: 0.8067 Epoch 9/32 471/471 [==============================] - 3s 7ms/step - loss: 1.2299 - accuracy: 0.8430 - val_loss: 1.3420 - val_accuracy: 0.8157 Epoch 10/32 471/471 [==============================] - 3s 7ms/step - loss: 1.1631 - accuracy: 0.8562 - val_loss: 1.2985 - val_accuracy: 0.8317 Epoch 11/32 471/471 [==============================] - 3s 6ms/step - loss: 1.1317 - accuracy: 0.8591 - val_loss: 1.3221 - val_accuracy: 0.8125 Epoch 12/32 471/471 [==============================] - 3s 6ms/step - loss: 1.0668 - accuracy: 0.8733 - val_loss: 1.2574 - val_accuracy: 0.8152 Epoch 13/32 471/471 [==============================] - 3s 6ms/step - loss: 1.0273 - accuracy: 0.8768 - val_loss: 1.2369 - val_accuracy: 0.8152 Epoch 14/32 471/471 [==============================] - 3s 7ms/step - loss: 0.9875 - accuracy: 0.8857 - val_loss: 1.1895 - val_accuracy: 0.8370 Epoch 15/32 471/471 [==============================] - 3s 6ms/step - loss: 0.9527 - accuracy: 0.8876 - val_loss: 1.1637 - val_accuracy: 0.8354 Epoch 16/32 471/471 [==============================] - 3s 6ms/step - loss: 0.9021 - accuracy: 0.9034 - val_loss: 1.1542 - val_accuracy: 0.8242 Epoch 17/32 471/471 [==============================] - 3s 7ms/step - loss: 0.8793 - accuracy: 0.9017 - val_loss: 1.1104 - val_accuracy: 0.8407 Epoch 18/32 471/471 [==============================] - 3s 6ms/step - loss: 0.8400 - accuracy: 0.9082 - val_loss: 1.1115 - val_accuracy: 0.8348 Epoch 19/32 471/471 [==============================] - 3s 7ms/step - loss: 0.8126 - accuracy: 0.9134 - val_loss: 1.0436 - val_accuracy: 0.8529 Epoch 20/32 471/471 [==============================] - 3s 6ms/step - loss: 0.7765 - accuracy: 0.9173 - val_loss: 1.0245 - val_accuracy: 0.8492 Epoch 21/32 471/471 [==============================] - 3s 6ms/step - loss: 0.7497 - accuracy: 0.9251 - val_loss: 1.0165 - val_accuracy: 0.8417 Epoch 22/32 471/471 [==============================] - 3s 6ms/step - loss: 0.7242 - accuracy: 0.9265 - val_loss: 1.0031 - val_accuracy: 0.8401 Epoch 23/32 471/471 [==============================] - 3s 6ms/step - loss: 0.7023 - accuracy: 0.9319 - val_loss: 0.9680 - val_accuracy: 0.8492 Epoch 24/32 471/471 [==============================] - 3s 6ms/step - loss: 0.6798 - accuracy: 0.9317 - val_loss: 0.9730 - val_accuracy: 0.8492 Epoch 25/32 471/471 [==============================] - 3s 7ms/step - loss: 0.6515 - accuracy: 0.9399 - val_loss: 0.9172 - val_accuracy: 0.8540 Epoch 26/32 471/471 [==============================] - 3s 6ms/step - loss: 0.6327 - accuracy: 0.9416 - val_loss: 0.9294 - val_accuracy: 0.8497 Epoch 27/32 471/471 [==============================] - 3s 7ms/step - loss: 0.6098 - accuracy: 0.9449 - val_loss: 0.8966 - val_accuracy: 0.8545 Epoch 28/32 471/471 [==============================] - 3s 6ms/step - loss: 0.5926 - accuracy: 0.9498 - val_loss: 0.9034 - val_accuracy: 0.8502 Epoch 29/32 471/471 [==============================] - 3s 7ms/step - loss: 0.5732 - accuracy: 0.9521 - val_loss: 0.8756 - val_accuracy: 0.8593 Epoch 30/32 471/471 [==============================] - 3s 6ms/step - loss: 0.5548 - accuracy: 0.9559 - val_loss: 0.8569 - val_accuracy: 0.8566 Epoch 31/32 471/471 [==============================] - 3s 6ms/step - loss: 0.5418 - accuracy: 0.9600 - val_loss: 0.8723 - val_accuracy: 0.8524 Epoch 32/32 471/471 [==============================] - 3s 7ms/step - loss: 0.5242 - accuracy: 0.9612 - val_loss: 0.8489 - val_accuracy: 0.8614 Trial 87 Complete [00h 01m 39s] val_accuracy: 0.8613914251327515 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 34m 15s Search: Running Trial #88 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 512 |512 |width none |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.5 |0.1 |dropout 256 |32 |batch_size adam |sgd |optimizer 0.01 |0.01 |learning_rate exponential |exponential |learning_rate_schedule 32 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/32 59/59 [==============================] - 4s 59ms/step - loss: 2.1386 - accuracy: 0.4718 - val_loss: 1.2910 - val_accuracy: 0.7095 Epoch 2/32 59/59 [==============================] - 3s 57ms/step - loss: 0.8335 - accuracy: 0.8176 - val_loss: 0.6700 - val_accuracy: 0.8311 Epoch 3/32 59/59 [==============================] - 4s 61ms/step - loss: 0.4165 - accuracy: 0.9093 - val_loss: 0.4947 - val_accuracy: 0.8704 Epoch 4/32 59/59 [==============================] - 4s 64ms/step - loss: 0.2466 - accuracy: 0.9543 - val_loss: 0.4239 - val_accuracy: 0.8874 Epoch 5/32 59/59 [==============================] - 4s 62ms/step - loss: 0.1590 - accuracy: 0.9732 - val_loss: 0.3891 - val_accuracy: 0.8986 Epoch 6/32 59/59 [==============================] - 3s 59ms/step - loss: 0.1076 - accuracy: 0.9852 - val_loss: 0.3751 - val_accuracy: 0.9044 Epoch 7/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0772 - accuracy: 0.9913 - val_loss: 0.3637 - val_accuracy: 0.9018 Epoch 8/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0561 - accuracy: 0.9946 - val_loss: 0.3614 - val_accuracy: 0.9044 Epoch 9/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0439 - accuracy: 0.9969 - val_loss: 0.3608 - val_accuracy: 0.9044 Epoch 10/32 59/59 [==============================] - 4s 60ms/step - loss: 0.0346 - accuracy: 0.9973 - val_loss: 0.3577 - val_accuracy: 0.9065 Epoch 11/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0285 - accuracy: 0.9982 - val_loss: 0.3580 - val_accuracy: 0.9033 Epoch 12/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0231 - accuracy: 0.9987 - val_loss: 0.3587 - val_accuracy: 0.9060 Epoch 13/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0197 - accuracy: 0.9989 - val_loss: 0.3605 - val_accuracy: 0.9055 Epoch 14/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0178 - accuracy: 0.9990 - val_loss: 0.3628 - val_accuracy: 0.9055 Epoch 15/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0155 - accuracy: 0.9992 - val_loss: 0.3650 - val_accuracy: 0.9060 Epoch 16/32 59/59 [==============================] - 4s 66ms/step - loss: 0.0137 - accuracy: 0.9993 - val_loss: 0.3660 - val_accuracy: 0.9076 Epoch 17/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0127 - accuracy: 0.9991 - val_loss: 0.3697 - val_accuracy: 0.9055 Epoch 18/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0113 - accuracy: 0.9991 - val_loss: 0.3703 - val_accuracy: 0.9055 Epoch 19/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0103 - accuracy: 0.9994 - val_loss: 0.3705 - val_accuracy: 0.9033 Epoch 20/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0098 - accuracy: 0.9992 - val_loss: 0.3729 - val_accuracy: 0.9049 Epoch 21/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0093 - accuracy: 0.9993 - val_loss: 0.3768 - val_accuracy: 0.9033 Epoch 22/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0086 - accuracy: 0.9993 - val_loss: 0.3790 - val_accuracy: 0.9033 Epoch 23/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0080 - accuracy: 0.9993 - val_loss: 0.3763 - val_accuracy: 0.9028 Epoch 24/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0075 - accuracy: 0.9992 - val_loss: 0.3780 - val_accuracy: 0.9039 Epoch 25/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0077 - accuracy: 0.9993 - val_loss: 0.3806 - val_accuracy: 0.9039 Epoch 26/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0067 - accuracy: 0.9994 - val_loss: 0.3813 - val_accuracy: 0.9028 Epoch 27/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0065 - accuracy: 0.9993 - val_loss: 0.3839 - val_accuracy: 0.9033 Epoch 28/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0062 - accuracy: 0.9992 - val_loss: 0.3830 - val_accuracy: 0.9018 Epoch 29/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0062 - accuracy: 0.9993 - val_loss: 0.3856 - val_accuracy: 0.9028 Epoch 30/32 59/59 [==============================] - 3s 55ms/step - loss: 0.0059 - accuracy: 0.9993 - val_loss: 0.3882 - val_accuracy: 0.9039 Epoch 31/32 59/59 [==============================] - 3s 56ms/step - loss: 0.0053 - accuracy: 0.9997 - val_loss: 0.3877 - val_accuracy: 0.9039 Epoch 32/32 59/59 [==============================] - 3s 56ms/step - loss: 0.0054 - accuracy: 0.9993 - val_loss: 0.3874 - val_accuracy: 0.9023 Trial 88 Complete [00h 01m 48s] val_accuracy: 0.9075942635536194 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 36m 03s Search: Running Trial #89 Value |Best Value So Far |Hyperparameter initialized |initialized |embedding_type 64 |512 |width l1 |l1_l2 |regularization_function 1e-05 |0.0001 |regularization_penalty 0.4 |0.1 |dropout 512 |32 |batch_size sgd |sgd |optimizer 0.001 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 32 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/32 30/30 [==============================] - 4s 109ms/step - loss: 2.9611 - accuracy: 0.1059 - val_loss: 2.9351 - val_accuracy: 0.1551 Epoch 2/32 30/30 [==============================] - 3s 112ms/step - loss: 2.8831 - accuracy: 0.2641 - val_loss: 2.8359 - val_accuracy: 0.3606 Epoch 3/32 30/30 [==============================] - 4s 120ms/step - loss: 2.8194 - accuracy: 0.3835 - val_loss: 2.8133 - val_accuracy: 0.3967 Epoch 4/32 30/30 [==============================] - 4s 121ms/step - loss: 2.7570 - accuracy: 0.4108 - val_loss: 2.6962 - val_accuracy: 0.4849 Epoch 5/32 30/30 [==============================] - 4s 121ms/step - loss: 2.6451 - accuracy: 0.5100 - val_loss: 2.6021 - val_accuracy: 0.5231 Epoch 6/32 30/30 [==============================] - 3s 115ms/step - loss: 2.5688 - accuracy: 0.5255 - val_loss: 2.5559 - val_accuracy: 0.5295 Epoch 7/32 30/30 [==============================] - 3s 108ms/step - loss: 2.5419 - accuracy: 0.5352 - val_loss: 2.5485 - val_accuracy: 0.5316 Epoch 8/32 30/30 [==============================] - 3s 112ms/step - loss: 2.4715 - accuracy: 0.5602 - val_loss: 2.4039 - val_accuracy: 0.5720 Epoch 9/32 30/30 [==============================] - 3s 106ms/step - loss: 2.3227 - accuracy: 0.5917 - val_loss: 2.2643 - val_accuracy: 0.6001 Epoch 10/32 30/30 [==============================] - 3s 115ms/step - loss: 2.1842 - accuracy: 0.6380 - val_loss: 2.1451 - val_accuracy: 0.6245 Epoch 11/32 30/30 [==============================] - 3s 115ms/step - loss: 2.0707 - accuracy: 0.6494 - val_loss: 2.0533 - val_accuracy: 0.6426 Epoch 12/32 30/30 [==============================] - 4s 121ms/step - loss: 1.9899 - accuracy: 0.6696 - val_loss: 1.9931 - val_accuracy: 0.6543 Epoch 13/32 30/30 [==============================] - 3s 115ms/step - loss: 1.9382 - accuracy: 0.6800 - val_loss: 1.9592 - val_accuracy: 0.6575 Epoch 14/32 30/30 [==============================] - 3s 107ms/step - loss: 1.9135 - accuracy: 0.6848 - val_loss: 1.9464 - val_accuracy: 0.6633 Epoch 15/32 30/30 [==============================] - 4s 121ms/step - loss: 1.9050 - accuracy: 0.6878 - val_loss: 1.9445 - val_accuracy: 0.6638 Epoch 16/32 30/30 [==============================] - 4s 120ms/step - loss: 1.8395 - accuracy: 0.6994 - val_loss: 1.8052 - val_accuracy: 0.6845 Epoch 17/32 30/30 [==============================] - 3s 115ms/step - loss: 1.6946 - accuracy: 0.7187 - val_loss: 1.6762 - val_accuracy: 0.7042 Epoch 18/32 30/30 [==============================] - 5s 173ms/step - loss: 1.5621 - accuracy: 0.7451 - val_loss: 1.5591 - val_accuracy: 0.7249 Epoch 19/32 30/30 [==============================] - 3s 114ms/step - loss: 1.4420 - accuracy: 0.7603 - val_loss: 1.4576 - val_accuracy: 0.7456 Epoch 20/32 30/30 [==============================] - 4s 121ms/step - loss: 1.3397 - accuracy: 0.7788 - val_loss: 1.3711 - val_accuracy: 0.7573 Epoch 21/32 30/30 [==============================] - 4s 121ms/step - loss: 1.2522 - accuracy: 0.7968 - val_loss: 1.3006 - val_accuracy: 0.7685 Epoch 22/32 30/30 [==============================] - 4s 122ms/step - loss: 1.1802 - accuracy: 0.8087 - val_loss: 1.2419 - val_accuracy: 0.7722 Epoch 23/32 30/30 [==============================] - 3s 115ms/step - loss: 1.1220 - accuracy: 0.8141 - val_loss: 1.1957 - val_accuracy: 0.7785 Epoch 24/32 30/30 [==============================] - 3s 108ms/step - loss: 1.0762 - accuracy: 0.8238 - val_loss: 1.1605 - val_accuracy: 0.7817 Epoch 25/32 30/30 [==============================] - 5s 163ms/step - loss: 1.0402 - accuracy: 0.8268 - val_loss: 1.1336 - val_accuracy: 0.7823 Epoch 26/32 30/30 [==============================] - 4s 121ms/step - loss: 1.0139 - accuracy: 0.8340 - val_loss: 1.1145 - val_accuracy: 0.7849 Epoch 27/32 30/30 [==============================] - 3s 112ms/step - loss: 0.9949 - accuracy: 0.8351 - val_loss: 1.1016 - val_accuracy: 0.7870 Epoch 28/32 30/30 [==============================] - 3s 104ms/step - loss: 0.9841 - accuracy: 0.8373 - val_loss: 1.0936 - val_accuracy: 0.7865 Epoch 29/32 30/30 [==============================] - 4s 121ms/step - loss: 0.9775 - accuracy: 0.8385 - val_loss: 1.0894 - val_accuracy: 0.7876 Epoch 30/32 30/30 [==============================] - 3s 115ms/step - loss: 0.9735 - accuracy: 0.8387 - val_loss: 1.0879 - val_accuracy: 0.7881 Epoch 31/32 30/30 [==============================] - 3s 102ms/step - loss: 0.9724 - accuracy: 0.8392 - val_loss: 1.0877 - val_accuracy: 0.7881 Epoch 32/32 30/30 [==============================] - 3s 113ms/step - loss: 0.9411 - accuracy: 0.8438 - val_loss: 1.0230 - val_accuracy: 0.7977 Trial 89 Complete [00h 01m 54s] val_accuracy: 0.7976633310317993 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 37m 56s Search: Running Trial #90 Value |Best Value So Far |Hyperparameter frozen |initialized |embedding_type 128 |512 |width l2 |l1_l2 |regularization_function 0.0001 |0.0001 |regularization_penalty 0.3 |0.1 |dropout 256 |32 |batch_size adam |sgd |optimizer 0.01 |0.01 |learning_rate cosine |exponential |learning_rate_schedule 32 |11 |tuner/epochs 0 |4 |tuner/initial_epoch 0 |2 |tuner/bracket 0 |1 |tuner/round Epoch 1/32 59/59 [==============================] - 2s 30ms/step - loss: 2.6349 - accuracy: 0.3334 - val_loss: 2.4771 - val_accuracy: 0.4073 Epoch 2/32 59/59 [==============================] - 2s 30ms/step - loss: 2.2163 - accuracy: 0.4819 - val_loss: 2.0129 - val_accuracy: 0.5725 Epoch 3/32 59/59 [==============================] - 2s 27ms/step - loss: 1.9645 - accuracy: 0.5671 - val_loss: 1.9378 - val_accuracy: 0.5661 Epoch 4/32 59/59 [==============================] - 2s 29ms/step - loss: 1.8090 - accuracy: 0.5817 - val_loss: 1.6767 - val_accuracy: 0.6097 Epoch 5/32 59/59 [==============================] - 2s 29ms/step - loss: 1.6118 - accuracy: 0.6181 - val_loss: 1.5505 - val_accuracy: 0.6325 Epoch 6/32 59/59 [==============================] - 2s 28ms/step - loss: 1.5203 - accuracy: 0.6408 - val_loss: 1.4971 - val_accuracy: 0.6405 Epoch 7/32 59/59 [==============================] - 2s 28ms/step - loss: 1.4913 - accuracy: 0.6496 - val_loss: 1.4877 - val_accuracy: 0.6484 Epoch 8/32 59/59 [==============================] - 2s 29ms/step - loss: 1.4453 - accuracy: 0.6417 - val_loss: 1.3828 - val_accuracy: 0.6580 Epoch 9/32 59/59 [==============================] - 2s 29ms/step - loss: 1.3458 - accuracy: 0.6583 - val_loss: 1.3032 - val_accuracy: 0.6660 Epoch 10/32 59/59 [==============================] - 2s 28ms/step - loss: 1.2824 - accuracy: 0.6656 - val_loss: 1.2590 - val_accuracy: 0.6665 Epoch 11/32 59/59 [==============================] - 2s 28ms/step - loss: 1.2340 - accuracy: 0.6859 - val_loss: 1.2164 - val_accuracy: 0.6840 Epoch 12/32 59/59 [==============================] - 2s 28ms/step - loss: 1.2037 - accuracy: 0.6893 - val_loss: 1.1978 - val_accuracy: 0.6872 Epoch 13/32 59/59 [==============================] - 2s 27ms/step - loss: 1.1862 - accuracy: 0.6937 - val_loss: 1.1832 - val_accuracy: 0.6845 Epoch 14/32 59/59 [==============================] - 2s 28ms/step - loss: 1.1794 - accuracy: 0.6958 - val_loss: 1.1795 - val_accuracy: 0.6845 Epoch 15/32 59/59 [==============================] - 2s 28ms/step - loss: 1.1733 - accuracy: 0.6969 - val_loss: 1.1789 - val_accuracy: 0.6856 Epoch 16/32 59/59 [==============================] - 2s 27ms/step - loss: 1.1636 - accuracy: 0.6881 - val_loss: 1.1513 - val_accuracy: 0.6856 Epoch 17/32 59/59 [==============================] - 2s 30ms/step - loss: 1.1320 - accuracy: 0.6920 - val_loss: 1.1155 - val_accuracy: 0.6888 Epoch 18/32 59/59 [==============================] - 2s 28ms/step - loss: 1.1022 - accuracy: 0.7010 - val_loss: 1.0861 - val_accuracy: 0.6925 Epoch 19/32 59/59 [==============================] - 2s 28ms/step - loss: 1.0736 - accuracy: 0.7107 - val_loss: 1.0641 - val_accuracy: 0.6973 Epoch 20/32 59/59 [==============================] - 2s 30ms/step - loss: 1.0550 - accuracy: 0.7097 - val_loss: 1.0419 - val_accuracy: 0.7026 Epoch 21/32 59/59 [==============================] - 2s 28ms/step - loss: 1.0345 - accuracy: 0.7143 - val_loss: 1.0275 - val_accuracy: 0.7047 Epoch 22/32 59/59 [==============================] - 2s 28ms/step - loss: 1.0196 - accuracy: 0.7176 - val_loss: 1.0194 - val_accuracy: 0.7058 Epoch 23/32 59/59 [==============================] - 2s 29ms/step - loss: 1.0055 - accuracy: 0.7218 - val_loss: 1.0165 - val_accuracy: 0.7074 Epoch 24/32 59/59 [==============================] - 2s 29ms/step - loss: 0.9969 - accuracy: 0.7227 - val_loss: 1.0011 - val_accuracy: 0.7079 Epoch 25/32 59/59 [==============================] - 2s 29ms/step - loss: 0.9875 - accuracy: 0.7238 - val_loss: 0.9979 - val_accuracy: 0.7100 Epoch 26/32 59/59 [==============================] - 2s 30ms/step - loss: 0.9839 - accuracy: 0.7276 - val_loss: 0.9920 - val_accuracy: 0.7111 Epoch 27/32 59/59 [==============================] - 2s 30ms/step - loss: 0.9813 - accuracy: 0.7254 - val_loss: 0.9900 - val_accuracy: 0.7153 Epoch 28/32 59/59 [==============================] - 2s 28ms/step - loss: 0.9744 - accuracy: 0.7289 - val_loss: 0.9873 - val_accuracy: 0.7100 Epoch 29/32 59/59 [==============================] - 2s 27ms/step - loss: 0.9715 - accuracy: 0.7300 - val_loss: 0.9853 - val_accuracy: 0.7132 Epoch 30/32 59/59 [==============================] - 2s 27ms/step - loss: 0.9708 - accuracy: 0.7348 - val_loss: 0.9853 - val_accuracy: 0.7116 Epoch 31/32 59/59 [==============================] - 2s 27ms/step - loss: 0.9709 - accuracy: 0.7326 - val_loss: 0.9853 - val_accuracy: 0.7127 Epoch 32/32 59/59 [==============================] - 2s 27ms/step - loss: 0.9790 - accuracy: 0.7249 - val_loss: 0.9799 - val_accuracy: 0.7090 Trial 90 Complete [00h 00m 55s] val_accuracy: 0.7153478264808655 Best val_accuracy So Far: 0.9081253409385681 Total elapsed time: 00h 38m 51s Results summary Results in tuning/bandit Showing 10 best trials Trial summary Hyperparameters: embedding_type: initialized width: 512 regularization_function: l1_l2 regularization_penalty: 0.0001 dropout: 0.1 batch_size: 32 optimizer: sgd learning_rate: 0.01 learning_rate_schedule: exponential tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 2 tuner/round: 1 tuner/trial_id: 0056 Score: 0.9081253409385681 Trial summary Hyperparameters: embedding_type: initialized width: 512 regularization_function: none regularization_penalty: 1e-05 dropout: 0.5 batch_size: 256 optimizer: adam learning_rate: 0.01 learning_rate_schedule: exponential tuner/epochs: 32 tuner/initial_epoch: 0 tuner/bracket: 0 tuner/round: 0 Score: 0.9075942635536194 Trial summary Hyperparameters: embedding_type: initialized width: 512 regularization_function: l1 regularization_penalty: 1e-05 dropout: 0.2 batch_size: 64 optimizer: sgd learning_rate: 0.01 learning_rate_schedule: none tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 3 tuner/round: 2 tuner/trial_id: 0035 Score: 0.9070631861686707 Trial summary Hyperparameters: embedding_type: initialized width: 512 regularization_function: l1 regularization_penalty: 1e-05 dropout: 0.2 batch_size: 64 optimizer: sgd learning_rate: 0.01 learning_rate_schedule: none tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 3 tuner/round: 3 tuner/trial_id: 0046 Score: 0.9065321087837219 Trial summary Hyperparameters: embedding_type: initialized width: 512 regularization_function: l1_l2 regularization_penalty: 0.0001 dropout: 0.1 batch_size: 32 optimizer: sgd learning_rate: 0.01 learning_rate_schedule: exponential tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 2 tuner/round: 2 tuner/trial_id: 0067 Score: 0.9054700136184692 Trial summary Hyperparameters: embedding_type: initialized width: 128 regularization_function: l1_l2 regularization_penalty: 0.0001 dropout: 0.2 batch_size: 32 optimizer: rmsprop learning_rate: 0.01 learning_rate_schedule: none tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 3 tuner/round: 2 tuner/trial_id: 0034 Score: 0.9049389362335205 Trial summary Hyperparameters: embedding_type: uninitialized width: 512 regularization_function: none regularization_penalty: 0.0001 dropout: 0.1 batch_size: 512 optimizer: adam learning_rate: 0.01 learning_rate_schedule: exponential tuner/epochs: 11 tuner/initial_epoch: 4 tuner/bracket: 3 tuner/round: 2 tuner/trial_id: 0037 Score: 0.9028146862983704 Trial summary Hyperparameters: embedding_type: initialized width: 512 regularization_function: l1_l2 regularization_penalty: 0.0001 dropout: 0.1 batch_size: 32 optimizer: sgd learning_rate: 0.01 learning_rate_schedule: exponential tuner/epochs: 4 tuner/initial_epoch: 0 tuner/bracket: 2 tuner/round: 0 Score: 0.9028146862983704 Trial summary Hyperparameters: embedding_type: initialized width: 64 regularization_function: l1_l2 regularization_penalty: 0.0001 dropout: 0.30000000000000004 batch_size: 32 optimizer: rmsprop learning_rate: 0.001 learning_rate_schedule: cosine tuner/epochs: 32 tuner/initial_epoch: 11 tuner/bracket: 1 tuner/round: 1 tuner/trial_id: 0077 Score: 0.9022836089134216 Trial summary Hyperparameters: embedding_type: initialized width: 128 regularization_function: l1_l2 regularization_penalty: 0.0001 dropout: 0.2 batch_size: 32 optimizer: rmsprop learning_rate: 0.01 learning_rate_schedule: none tuner/epochs: 2 tuner/initial_epoch: 0 tuner/bracket: 3 tuner/round: 0 Score: 0.9006903767585754 {'embedding_type': 'initialized', 'width': 512, 'regularization_function': 'l1_l2', 'regularization_penalty': 0.0001, 'dropout': 0.1, 'batch_size': 32, 'optimizer': 'sgd', 'learning_rate': 0.01, 'learning_rate_schedule': 'exponential', 'tuner/epochs': 11, 'tuner/initial_epoch': 4, 'tuner/bracket': 2, 'tuner/round': 1, 'tuner/trial_id': '0056'} 59/59 [==============================] - 0s 2ms/step Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= embedding (Embedding) (None, 948, 300) 4680900 dropout (Dropout) (None, 948, 300) 0 global_average_pooling1d (G (None, 300) 0 lobalAveragePooling1D) dense (Dense) (None, 20) 6020 ================================================================= Total params: 4,686,920 Trainable params: 4,686,920 Non-trainable params: 0 _________________________________________________________________ real 39m31.442s user 19m33.686s sys 2m16.842s deeplearn@ML-RefVm-967342:~/newsgroups$ kaggle competitions submit -c ml530-2022-fall-newsgroups -f predictions.csv -m "39:31" 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 15.6k/15.6k [00:00<00:00, 37.5kB/s] Successfully submitted to ml530-2022-fall-newsgroupsdeeplearn@ML-RefVm-967342:~/newsgroups$