Im Load From Pickle but Train Again

How to Relieve and Load Your Keras Deep Learning Model

Last Updated on August 27, 2020

Keras is a unproblematic and powerful Python library for deep learning.

Given that deep learning models can take hours, days and even weeks to train, it is of import to know how to salvage and load them from deejay.

In this post, you will discover how you can save your Keras models to file and load them up once again to brand predictions.

After reading this tutorial you volition know:

• How to save model weights and model architecture in dissever files.
• How to save model architecture in both YAML and JSON format.
• How to save model weights and architecture into a single file for later use.

Kick-offset your project with my new book Deep Learning With Python, including step-by-step tutorials and the Python source lawmaking files for all examples.

Let's get started.

• Update Mar 2017: Added instructions to install h5py first.
• Update Mar/2017: Updated examples for changes to the Keras API.
• Update Mar/2018: Added alternate link to download the dataset.
• Update May/2019: Added department on saving and loading the model to a unmarried file.
• Update Sep/2019: Added note about using PyYAML version 5.

How to Salve and Load Your Keras Deep Learning Models
Photograph past art_inthecity, some rights reserved.

Tutorial Overview

If y'all are new to Keras or deep learning, run into this step-past-step Keras tutorial.

Keras separates the concerns of saving your model architecture and saving your model weights.

Model weights are saved to HDF5 format. This is a grid format that is ideal for storing multi-dimensional arrays of numbers.

The model structure tin be described and saved using two different formats: JSON and YAML.

In this post nosotros are going to look at ii examples of saving and loading your model to file:

• Relieve Model to JSON.
• Save Model to YAML.

Each example volition too demonstrate saving and loading your model weights to HDF5 formatted files.

The examples will use the same simple network trained on the Pima Indians onset of diabetes binary classification dataset. This is a small dataset that contains all numerical information and is easy to work with. You can download this dataset and place it in your working directory with the filename "pima-indians-diabetes.csv" (update: download from here).

Confirm that yous have the latest version of Keras installed (east.g. v2.2.iv as of May 2019).

Notation: Saving models requires that y'all have the h5py library installed. Y'all tin install information technology easily as follows:

Demand help with Deep Learning in Python?

Accept my free 2-calendar week email class and discover MLPs, CNNs and LSTMs (with code).

Click to sign-up now and also get a gratuitous PDF Ebook version of the grade.

Save Your Neural Network Model to JSON

JSON is a unproblematic file format for describing information hierarchically.

Keras provides the ability to describe any model using JSON format with a to_json() part. This can be saved to file and afterwards loaded via the model_from_json() role that will create a new model from the JSON specification.

The weights are saved directly from the model using the save_weights() part and later on loaded using the symmetrical load_weights() part.

The example below trains and evaluates a unproblematic model on the Pima Indians dataset. The model is so converted to JSON format and written to model.json in the local directory. The network weights are written to model.h5 in the local directory.

The model and weight information is loaded from the saved files and a new model is created. Information technology is important to compile the loaded model before information technology is used. This is and so that predictions made using the model tin can use the appropriate efficient computation from the Keras backend.

The model is evaluated in the same way printing the same evaluation score.

i ii 3 4 5 6 vii 8 9 10 xi 12 13 fourteen 15 16 17 xviii 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 twoscore 41 42 43 44 45 46 47 48 49

# MLP for Pima Indians Dataset Serialize to JSON and HDF5 from keras . models import Sequential from keras . layers import Dense from keras . models import model_from_json import numpy import os # fix random seed for reproducibility numpy . random . seed ( 7 ) # load pima indians dataset dataset = numpy . loadtxt ( "pima-indians-diabetes.csv" , delimiter = "," ) # split into input (X) and output (Y) variables Ten = dataset [ : , 0 : 8 ] Y = dataset [ : , viii ] # create model model = Sequential ( ) model . add ( Dense ( 12 , input_dim = eight , activation = 'relu' ) ) model . add ( Dense ( 8 , activation = 'relu' ) ) model . add ( Dense ( ane , activation = 'sigmoid' ) ) # Compile model model . compile ( loss = 'binary_crossentropy' , optimizer = 'adam' , metrics = [ 'accuracy' ] ) # Fit the model model . fit ( X , Y , epochs = 150 , batch_size = 10 , verbose = 0 ) # evaluate the model scores = model . evaluate ( Ten , Y , verbose = 0 ) print ( "%due south: %.2f%%" % ( model . metrics_names [ 1 ] , scores [ 1 ] * 100 ) ) # serialize model to JSON model_json = model . to_json ( ) with open ( "model.json" , "w" ) as json_file : json_file . write ( model_json ) # serialize weights to HDF5 model . save_weights ( "model.h5" ) print ( "Saved model to deejay" ) # later... # load json and create model json_file = open ( 'model.json' , 'r' ) loaded_model_json = json_file . read ( ) json_file . close ( ) loaded_model = model_from_json ( loaded_model_json ) # load weights into new model loaded_model . load_weights ( "model.h5" ) print ( "Loaded model from disk" ) # evaluate loaded model on test data loaded_model . compile ( loss = 'binary_crossentropy' , optimizer = 'rmsprop' , metrics = [ 'accuracy' ] ) score = loaded_model . evaluate ( X , Y , verbose = 0 ) print ( "%s: %.2f%%" % ( loaded_model . metrics_names [ i ] , score [ one ] * 100 ) )

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

Running this example provides the output below.

acc: 78.78% Saved model to deejay Loaded model from disk acc: 78.78%

The JSON format of the model looks like the following:

1 ii 3 4 5 6 7 viii 9 10 11 12 13 14 xv 16 17 xviii 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 l 51 52 53 54 55 56 57 58 59 sixty 61 62 63 64 65 66 67 68 69 lxx 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107

{    "class_name":"Sequential",    "config":{       "proper name":"sequential_1",       "layers":[          {             "class_name":"Dumbo",             "config":{                "name":"dense_1",                "trainable":true,                "batch_input_shape":[                   nix,                   8                ],                "dtype":"float32",                "units":12,                "activation":"relu",                "use_bias":true,                "kernel_initializer":{                   "class_name":"VarianceScaling",                   "config":{                      "scale":i.0,                      "fashion":"fan_avg",                      "distribution":"compatible",                      "seed":null                   }                },                "bias_initializer":{                   "class_name":"Zeros",                   "config":{                   }                },                "kernel_regularizer":null,                "bias_regularizer":cypher,                "activity_regularizer":nil,                "kernel_constraint":null,                "bias_constraint":goose egg             }          },          {             "class_name":"Dense",             "config":{                "name":"dense_2",                "trainable":truthful,                "dtype":"float32",                "units":8,                "activation":"relu",                "use_bias":true,                "kernel_initializer":{                   "class_name":"VarianceScaling",                   "config":{                      "calibration":1.0,                      "mode":"fan_avg",                      "distribution":"uniform",                      "seed":null                   }                },                "bias_initializer":{                   "class_name":"Zeros",                   "config":{                   }                },                "kernel_regularizer":aught,                "bias_regularizer":zilch,                "activity_regularizer":null,                "kernel_constraint":nada,                "bias_constraint":cypher             }          },          {             "class_name":"Dumbo",             "config":{                "proper noun":"dense_3",                "trainable":true,                "dtype":"float32",                "units":1,                "activation":"sigmoid",                "use_bias":true,                "kernel_initializer":{                   "class_name":"VarianceScaling",                   "config":{                      "calibration":1.0,                      "mode":"fan_avg",                      "distribution":"compatible",                      "seed":null                   }                },                "bias_initializer":{                   "class_name":"Zeros",                   "config":{                   }                },                "kernel_regularizer":null,                "bias_regularizer":null,                "activity_regularizer":null,                "kernel_constraint":naught,                "bias_constraint":null             }          }       ]    },    "keras_version":"2.ii.5",    "backend":"tensorflow" }

Salve Your Neural Network Model to YAML

This example is much the same as the to a higher place JSON example, except the YAML format is used for the model specification.

Note, this example assumes that you have PyYAML 5 installed, for instance:

In this example, the model is described using YAML, saved to file model.yaml and later loaded into a new model via the model_from_yaml() function.

Weights are handled in the aforementioned way as above in HDF5 format as model.h5.

1 2 iii four 5 six seven 8 9 x 11 12 13 14 fifteen xvi 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

# MLP for Pima Indians Dataset serialize to YAML and HDF5 from keras . models import Sequential from keras . layers import Dense from keras . models import model_from_yaml import numpy import os # set up random seed for reproducibility seed = vii numpy . random . seed ( seed ) # load pima indians dataset dataset = numpy . loadtxt ( "pima-indians-diabetes.csv" , delimiter = "," ) # split into input (Ten) and output (Y) variables Ten = dataset [ : , 0 : eight ] Y = dataset [ : , 8 ] # create model model = Sequential ( ) model . add ( Dumbo ( 12 , input_dim = viii , activation = 'relu' ) ) model . add together ( Dense ( eight , activation = 'relu' ) ) model . add ( Dense ( i , activation = 'sigmoid' ) ) # Compile model model . compile ( loss = 'binary_crossentropy' , optimizer = 'adam' , metrics = [ 'accuracy' ] ) # Fit the model model . fit ( X , Y , epochs = 150 , batch_size = ten , verbose = 0 ) # evaluate the model scores = model . evaluate ( X , Y , verbose = 0 ) print ( "%southward: %.2f%%" % ( model . metrics_names [ ane ] , scores [ 1 ] * 100 ) ) # serialize model to YAML model_yaml = model . to_yaml ( ) with open ( "model.yaml" , "w" ) as yaml_file : yaml_file . write ( model_yaml ) # serialize weights to HDF5 model . save_weights ( "model.h5" ) impress ( "Saved model to disk" ) # afterward... # load YAML and create model yaml_file = open ( 'model.yaml' , 'r' ) loaded_model_yaml = yaml_file . read ( ) yaml_file . shut ( ) loaded_model = model_from_yaml ( loaded_model_yaml ) # load weights into new model loaded_model . load_weights ( "model.h5" ) print ( "Loaded model from disk" ) # evaluate loaded model on test information loaded_model . compile ( loss = 'binary_crossentropy' , optimizer = 'rmsprop' , metrics = [ 'accurateness' ] ) score = loaded_model . evaluate ( X , Y , verbose = 0 ) print ( "%southward: %.2f%%" % ( loaded_model . metrics_names [ 1 ] , score [ 1 ] * 100 ) )

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the boilerplate outcome.

Running the case displays the following output.

acc: 78.78% Saved model to deejay Loaded model from disk acc: 78.78%

The model described in YAML format looks like the following:

1 2 3 4 5 6 seven viii 9 x 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 xxx 31 32 33 34 35 36 37 38 39 forty 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 threescore 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78

backend: tensorflow class_name: Sequential config:   layers:   - class_name: Dense     config:       activation: relu       activity_regularizer: null       batch_input_shape: !!python/tuple       - zip       - viii       bias_constraint: goose egg       bias_initializer:         class_name: Zeros         config: {}       bias_regularizer: null       dtype: float32       kernel_constraint: null       kernel_initializer:         class_name: VarianceScaling         config:           distribution: compatible           fashion: fan_avg           scale: 1.0           seed: null       kernel_regularizer: naught       name: dense_1       trainable: true       units: 12       use_bias: true   - class_name: Dumbo     config:       activation: relu       activity_regularizer: null       bias_constraint: nix       bias_initializer:         class_name: Zeros         config: {}       bias_regularizer: null       dtype: float32       kernel_constraint: null       kernel_initializer:         class_name: VarianceScaling         config:           distribution: uniform           way: fan_avg           scale: 1.0           seed: null       kernel_regularizer: zilch       name: dense_2       trainable: true       units: 8       use_bias: true   - class_name: Dense     config:       activation: sigmoid       activity_regularizer: null       bias_constraint: aught       bias_initializer:         class_name: Zeros         config: {}       bias_regularizer: nada       dtype: float32       kernel_constraint: zip       kernel_initializer:         class_name: VarianceScaling         config:           distribution: uniform           mode: fan_avg           scale: 1.0           seed: null       kernel_regularizer: naught       name: dense_3       trainable: true       units: 1       use_bias: true   proper name: sequential_1 keras_version: 2.two.5

Salve Model Weights and Compages Together

Keras likewise supports a simpler interface to save both the model weights and model architecture together into a unmarried H5 file.

Saving the model in this mode includes everything we demand to know nigh the model, including:

• Model weights.
• Model architecture.
• Model compilation details (loss and metrics).
• Model optimizer state.

This means that we tin can load and use the model directly, without having to re-compile it as nosotros did in the examples in a higher place.

Annotation: this is the preferred way for saving and loading your Keras model.

How to Save a Keras Model

You tin save your model by calling the salve() function on the model and specifying the filename.

The example beneath demonstrates this by get-go plumbing fixtures a model, evaluating it and saving information technology to the file model.h5.

1 2 3 4 5 half dozen 7 8 9 10 11 12 thirteen 14 15 16 17 eighteen 19 20 21 22 23 24

# MLP for Pima Indians Dataset saved to single file from numpy import loadtxt from keras . models import Sequential from keras . layers import Dense # load pima indians dataset dataset = loadtxt ( "pima-indians-diabetes.csv" , delimiter = "," ) # split into input (X) and output (Y) variables Ten = dataset [ : , 0 : 8 ] Y = dataset [ : , viii ] # define model model = Sequential ( ) model . add together ( Dense ( 12 , input_dim = 8 , activation = 'relu' ) ) model . add ( Dense ( 8 , activation = 'relu' ) ) model . add together ( Dumbo ( ane , activation = 'sigmoid' ) ) # compile model model . compile ( loss = 'binary_crossentropy' , optimizer = 'adam' , metrics = [ 'accuracy' ] ) # Fit the model model . fit ( X , Y , epochs = 150 , batch_size = 10 , verbose = 0 ) # evaluate the model scores = model . evaluate ( X , Y , verbose = 0 ) impress ( "%s: %.2f%%" % ( model . metrics_names [ i ] , scores [ 1 ] * 100 ) ) # save model and compages to single file model . save ( "model.h5" ) print ( "Saved model to deejay" )

Note: Your results may vary given the stochastic nature of the algorithm or evaluation process, or differences in numerical precision. Consider running the example a few times and compare the boilerplate outcome.

Running the example fits the model, summarizes the models performance on the training dataset and saves the model to file.

acc: 77.73% Saved model to deejay

We tin later load this model from file and use it.

How to Load a Keras Model

Your saved model tin then exist loaded later by calling the load_model() function and passing the filename. The function returns the model with the aforementioned compages and weights.

In this case, we load the model, summarize the architecture and evaluate it on the same dataset to ostend the weights and architecture are the same.

# load and evaluate a saved model from numpy import loadtxt from keras . models import load _model # load model model = load_model ( 'model.h5' ) # summarize model. model . summary ( ) # load dataset dataset = loadtxt ( "pima-indians-diabetes.csv" , delimiter = "," ) # split into input (Ten) and output (Y) variables X = dataset [ : , 0 : 8 ] Y = dataset [ : , 8 ] # evaluate the model score = model . evaluate ( X , Y , verbose = 0 ) print ( "%s: %.2f%%" % ( model . metrics_names [ 1 ] , score [ 1 ] * 100 ) )

Running the case commencement loads the model, prints a summary of the model compages so evaluates the loaded model on the same dataset.

Note: Your results may vary given the stochastic nature of the algorithm or evaluation process, or differences in numerical precision. Consider running the example a few times and compare the average issue.

The model achieves the same accuracy score which in this example is 77%.

_________________________________________________________________ Layer (blazon)                 Output Shape              Param # ================================================================= dense_1 (Dense)              (None, 12)                108 _________________________________________________________________ dense_2 (Dumbo)              (None, eight)                 104 _________________________________________________________________ dense_3 (Dumbo)              (None, 1)                 9 ================================================================= Full params: 221 Trainable params: 221 Non-trainable params: 0 _________________________________________________________________ acc: 77.73%

Farther Reading

• How can I salvage a Keras model? in the Keras documentation.
• About Keras models in the Keras documentation.

Summary

In this post, yous discovered how to serialize your Keras deep learning models.

You learned how you can salvage your trained models to files and after load them upwardly and use them to make predictions.

Y'all also learned that model weights are hands stored using  HDF5 format and that the network structure can be saved in either JSON or YAML format.

Exercise you have whatsoever questions most saving your deep learning models or virtually this mail service?
Ask your questions in the comments and I will do my best to answer them.

Develop Deep Learning Projects with Python!

 What If Y'all Could Develop A Network in Minutes

...with but a few lines of Python

Discover how in my new Ebook:
Deep Learning With Python

It covers stop-to-terminate projects on topics like:
Multilayer Perceptrons,Convolutional Nets andRecurrent Neural Nets, and more...

Finally Bring Deep Learning To
Your Own Projects

Skip the Academics. Merely Results.

Run into What's Inside

chatmanmusly1995.blogspot.com

Source: https://machinelearningmastery.com/save-load-keras-deep-learning-models/

Share this post