背景
几个最简单的例子，展示一下 TensorFlow 的基本用法。
常量初体验
常量显示
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print sess.run(hello)
结果
Hello, TensorFlow!
常量加法
tensor
TensorFlow 的数据组成单元为 tensor，一个 tensor 由任意维度的数组组成，数组是由最小集合变换而来。
3 # 秩为零的张量 (a rank 0 tensor); a scalar with shape []
[1., 2., 3.] # 秩为一的张量 (a rank 1 tensor); a vector with shape [3]
[[1., 2., 3.], [4., 5., 6.]] # 秩为二的张量 (a rank 2 tensor); a matrix with shape [2, 3]
[[[1., 2., 3.]], [[7., 8., 9.]]] # 秩为三的张量 (a rank 3 tensor) with shape [2, 1, 3]
示例
这里做一个常量加法的示例
a = tf.constant(10)
b = tf.constant(20)
print sess.run(a+b)
输出
30
图初体验
计算图(Computational Graph)是将 TensorFlow 的操作放入图的结点中。
常量图
import tensorflow as tf
node1 = tf.constant(3.0, dtype=tf.float32)
node2 = tf.constant(4.0)
print(node1, node2)
输出
(<tf.Tensor 'Const:0' shape=() dtype=float32>, <tf.Tensor 'Const_1:0' shape=() dtype=float32>)
运行上面的图
sess = tf.Session()
print(sess.run([node1, node2]))
输出
[3.0, 4.0]
常量加法图
import tensorflow as tf
node1 = tf.constant(3.0, dtype=tf.float32)
node2 = tf.constant(4.0)
node3 = tf.add(node1, node2)
print("node3:", node3)
print("sess.run(node3):", sess.run(node3))
输出
>>> print("node3:", node3)
('node3:', <tf.Tensor 'Add:0' shape=() dtype=float32>)
>>> print("sess.run(node3):", sess.run(node3))
('sess.run(node3):', 7.0)
TensorBoard 来可视化图
记录日志
需要先把日志写到 logs 目录下，这里取上面的常量加法图的例子：
import tensorflow as tf
node1 = tf.constant(3.0, dtype=tf.float32)
node2 = tf.constant(4.0)
node3 = tf.add(node1, node2)
print("node3:", node3)
sess = tf.Session()
writer = tf.summary.FileWriter("logs/", sess.graph)
init = tf.global_variables_initializer()
print("sess.run(node3):", sess.run(node3))
这样发现日志文件 logs/events.out.tfevents.1549550259.abeffect，内容为二进制格式
可视化
通过 TensorBoard 来看到可视化的图
tensorboard --logdir logs
结果
TensorBoard 1.12.2 at http://abeffect:6006 (Press CTRL+C to quit)
打开浏览器，可见图：
变量加法
前面的图都是常量，这里通过占位符来实现变量的功能
声明占位符
a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
adder_node = a + b # + provides a shortcut for tf.add(a, b)
使用占位符
print(sess.run(adder_node, {a: 3, b: 4.5}))
print(sess.run(adder_node, {a: [1, 3], b: [2, 4]}))
输出
>>> print(sess.run(adder_node, {a: 3, b: 4.5}))
7.5
>>> print(sess.run(adder_node, {a: [1, 3], b: [2, 4]}))
[ 3. 7.]
变量乘法
稍等复杂一点的点位符操作，在变量加法的基础上，再进行乘法操作。
add_and_triple = adder_node * 3.
print(sess.run(add_and_triple, {a: 3, b: 4.5}))
输出
>>> print(sess.run(add_and_triple, {a: 3, b: 4.5}))
22.5
模型初体验
这里先建立一个线性模型 linear_model = W*x + b ，然后通过定义损失函数，对模型进行训练，最终计算出正确的 W 和 b 值。
建立线性模型
在变量的基础上，我们就可以来训练模型啦。
定义一个模型：linear_model = W*x + b
其中 W 的初值为 0.3, b 的初值为 -0.3，即模型初始状态为:
linear_model = W*x + b = 0.3 * x - 0.3
代码片断如下
import tensorflow as tf
W = tf.Variable([.3], dtype=tf.float32)
b = tf.Variable([-.3], dtype=tf.float32)
x = tf.placeholder(tf.float32)
linear_model = W*x + b
初始化所有的变量
sess = tf.Session()
writer = tf.summary.FileWriter("logs/", sess.graph)
init = tf.global_variables_initializer()
sess.run(init)
运行
print(sess.run(linear_model, {x: [1, 2, 3, 4]}))
输出
[ 0. 0.30000001 0.60000002 0.90000004]
定义损失函数
要评估模型的好坏，需要一个 y 变量来提供预期的值，然后编写损失函数 loss。
接着上面的例子，定义损失函数, 具体如下：
loss = tf.reduce_sum(tf.square( linear_model - y))
= tf.reduce_sum(tf.square( 0.3 * x - 0.3 - y))
代码如下:
y = tf.placeholder(tf.float32)
squared_deltas = tf.square(linear_model - y)
loss = tf.reduce_sum(squared_deltas)
print(sess.run(loss, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]}))
测试的 loss 函数的输出为:
>>> print(sess.run(loss, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]}))
23.66
手工验算，计算过程为：
>>> 1.3 * 1.3 + 2.6 * 2.6 + 3.9 * 3.9
23.66
由于这里 W 和 b 是程序初始化的值, 并不是正确的 W, b 值。为了减少 loss 值，我们先在损失函数中手动指定正确的 W 和 b 的值，然后通过模型训练来得到正确的 W, b 值。
这里正确的 W 值为 -1, b 值为 1，代码如下：
fixW = tf.assign(W, [-1.])
fixb = tf.assign(b, [1.])
sess.run([fixW, fixb])
print(sess.run(loss, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]}))
输出
>>> sess.run([fixW, fixb])
[array([-1.], dtype=float32), array([ 1.], dtype=float32)]
>>> print(sess.run(loss, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]}))
0.0
手工验算，计算过程为：
>>> 0 + 0 + 0 + 0
0
训练模型
TensorFlow 提供了优化方法，通过缓慢调整每一个变量的值，来实现最小化损失函数 loss。
这些优化方法在 tf.train api 中，其中最简单的优化算子为梯度下降算子，对应的 tf 方法为 tf.gradients。
代码示例
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
sess.run(init) # reset values to incorrect defaults.
for i in range(1000):
sess.run(train, {x: [1, 2, 3, 4], y: [0, -1, -2, -3]})
print(sess.run([W, b]))
输出
>>> print(sess.run([W, b]))
[array([-0.9999969], dtype=float32), array([ 0.99999082], dtype=float32)]
可见通过训练，得到了正确的值: W = -1, b = 1
完整代码
# -*- coding:utf-8 -*-
import tensorflow as tf
# 模型中参数的初始值 W = 0.3, b = -0.3
W = tf.Variable([.3], dtype=tf.float32)
b = tf.Variable([-.3], dtype=tf.float32)
# 模型变量 linear_model = W * x + b
x = tf.placeholder(tf.float32)
linear_model = W*x + b
# loss损失函数
y = tf.placeholder(tf.float32)
loss = tf.reduce_sum(tf.square(linear_model - y)) # 平方和
# 优化方法
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
# 样本数据
x_train = [1, 2, 3, 4]
y_train = [0, -1, -2, -3]
# 开始训练
init = tf.global_variables_initializer()
sess = tf.Session()
writer = tf.summary.FileWriter("logs/", sess.graph)
sess.run(init) # reset values to wrong
for i in range(1000):
sess.run(train, {x: x_train, y: y_train})
# 计算下准确度
curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train})
print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))
输出
W: [-0.9999969] b: [0.9999908] loss: 5.6999738e-11
可视化图
附录
tf.estimator
tf.estimator 是高级的 tf 库，来简化机器学习操作，包括
- 进行训练
- 进行估算
- 管理数据
同时包含很多常用的模型
入门
# NumPy is often used to load, manipulate and preprocess data.
import numpy as np
import tensorflow as tf
# Declare list of features. We only have one numeric feature. There are many
# other types of columns that are more complicated and useful.
feature_columns = [tf.feature_column.numeric_column("x", shape=[1])]
# An estimator is the front end to invoke training (fitting) and evaluation
# (inference). There are many predefined types like linear regression,
# linear classification, and many neural network classifiers and regressors.
# The following code provides an estimator that does linear regression.
estimator = tf.estimator.LinearRegressor(feature_columns=feature_columns)
# TensorFlow provides many helper methods to read and set up data sets.
# Here we use two data sets: one for training and one for evaluation
# We have to tell the function how many batches
# of data (num_epochs) we want and how big each batch should be.
x_train = np.array([1., 2., 3., 4.])
y_train = np.array([0., -1., -2., -3.])
x_eval = np.array([2., 5., 8., 1.])
y_eval = np.array([-1.01, -4.1, -7, 0.])
input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_train}, y_train, batch_size=4, num_epochs=None, shuffle=True)
train_input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_train}, y_train, batch_size=4, num_epochs=1000, shuffle=False)
eval_input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_eval}, y_eval, batch_size=4, num_epochs=1000, shuffle=False)
# We can invoke 1000 training steps by invoking the method and passing the
# training data set.
estimator.train(input_fn=input_fn, steps=1000)
# Here we evaluate how well our model did.
train_metrics = estimator.evaluate(input_fn=train_input_fn)
eval_metrics = estimator.evaluate(input_fn=eval_input_fn)
print("train metrics: %r"% train_metrics)
print("eval metrics: %r"% eval_metrics)
估算子输出
>>> estimator = tf.estimator.LinearRegressor(feature_columns=feature_columns)
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmp4Qnyls
INFO:tensorflow:Using config: {'_save_checkpoints_secs': 600, '_session_config': None, '_keep_checkpoint_max': 5, '_task_type': 'worker', '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x10905c3d0>, '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_service': None, '_num_ps_replicas': 0, '_tf_random_seed': None, '_master': '', '_num_worker_replicas': 1, '_task_id': 0, '_log_step_count_steps': 100, '_model_dir': '/var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmp4Qnyls', '_save_summary_steps': 100}
训练输出
>>> estimator.train(input_fn=input_fn, steps=1000)
INFO:tensorflow:Create CheckpointSaverHook.
2017-12-10 10:55:36.373972: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA
INFO:tensorflow:Saving checkpoints for 1 into /var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmp4Qnyls/model.ckpt.
INFO:tensorflow:loss = 14.0, step = 1
INFO:tensorflow:global_step/sec: 1063.92
INFO:tensorflow:loss = 0.269534, step = 101 (0.094 sec)
INFO:tensorflow:global_step/sec: 1066.04
INFO:tensorflow:loss = 0.0797396, step = 201 (0.094 sec)
INFO:tensorflow:global_step/sec: 1062.66
INFO:tensorflow:loss = 0.0232967, step = 301 (0.093 sec)
INFO:tensorflow:global_step/sec: 1237.36
INFO:tensorflow:loss = 0.00372493, step = 401 (0.081 sec)
INFO:tensorflow:global_step/sec: 1371.82
INFO:tensorflow:loss = 0.000646794, step = 501 (0.073 sec)
INFO:tensorflow:global_step/sec: 1410.69
INFO:tensorflow:loss = 0.000262144, step = 601 (0.071 sec)
INFO:tensorflow:global_step/sec: 1411.79
INFO:tensorflow:loss = 3.975e-05, step = 701 (0.071 sec)
INFO:tensorflow:global_step/sec: 1436.49
INFO:tensorflow:loss = 2.31565e-05, step = 801 (0.070 sec)
INFO:tensorflow:global_step/sec: 1358.66
INFO:tensorflow:loss = 2.64456e-06, step = 901 (0.074 sec)
INFO:tensorflow:Saving checkpoints for 1000 into /var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmp4Qnyls/model.ckpt.
INFO:tensorflow:Loss for final step: 3.15212e-07.
<tensorflow.python.estimator.canned.linear.LinearRegressor object at 0x100536a50>
训练集的 metrics
>>> train_metrics = estimator.evaluate(input_fn=train_input_fn)
INFO:tensorflow:Starting evaluation at 2017-12-10-02:56:18
INFO:tensorflow:Restoring parameters from /var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmp4Qnyls/model.ckpt-1000
INFO:tensorflow:Finished evaluation at 2017-12-10-02:56:18
INFO:tensorflow:Saving dict for global step 1000: average_loss = 1.19704e-07, global_step = 1000, loss = 4.78816e-07
评估集的 metrics
>>> eval_metrics = estimator.evaluate(input_fn=eval_input_fn)
INFO:tensorflow:Starting evaluation at 2017-12-10-02:56:39
INFO:tensorflow:Restoring parameters from /var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmp4Qnyls/model.ckpt-1000
INFO:tensorflow:Finished evaluation at 2017-12-10-02:56:40
INFO:tensorflow:Saving dict for global step 1000: average_loss = 0.00255552, global_step = 1000, loss = 0.0102221
>>> print("train metrics: %r"% train_metrics)
train metrics: {'average_loss': 1.1970396e-07, 'global_step': 1000, 'loss': 4.7881582e-07}
>>> print("eval metrics: %r"% eval_metrics)
eval metrics: {'average_loss': 0.0025555168, 'global_step': 1000, 'loss': 0.010222067}
自定义模型
tf.estimator 并没有限定只能使用内置的模型.
这里给一个使用自定义的 LinearRegressor 模型的例子，同时复用上面的主要逻辑.
使用自定义模型，关键在于 tf.estimator.Estimator 类的使用.
tf.estimator.LinearRegressor 正是类 tf.estimator.Estimator 的一个子类
所以我们只需要自定义方法 model_fn，来告诉 tf.estimator 如何评估预测，训练和损失函数. 示例如下
import numpy as np
import tensorflow as tf
# Declare list of features, we only have one real-valued feature
def model_fn(features, labels, mode):
# Build a linear model and predict values
W = tf.get_variable("W", [1], dtype=tf.float64)
b = tf.get_variable("b", [1], dtype=tf.float64)
y = W*features['x'] + b
# Loss sub-graph
loss = tf.reduce_sum(tf.square(y - labels))
# Training sub-graph
global_step = tf.train.get_global_step()
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = tf.group(optimizer.minimize(loss),
tf.assign_add(global_step, 1))
# EstimatorSpec connects subgraphs we built to the
# appropriate functionality.
return tf.estimator.EstimatorSpec(
mode=mode,
predictions=y,
loss=loss,
train_op=train)
estimator = tf.estimator.Estimator(model_fn=model_fn)
# define our data sets
x_train = np.array([1., 2., 3., 4.])
y_train = np.array([0., -1., -2., -3.])
x_eval = np.array([2., 5., 8., 1.])
y_eval = np.array([-1.01, -4.1, -7., 0.])
input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_train}, y_train, batch_size=4, num_epochs=None, shuffle=True)
train_input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_train}, y_train, batch_size=4, num_epochs=1000, shuffle=False)
eval_input_fn = tf.estimator.inputs.numpy_input_fn(
{"x": x_eval}, y_eval, batch_size=4, num_epochs=1000, shuffle=False)
# train
estimator.train(input_fn=input_fn, steps=1000)
# Here we evaluate how well our model did.
train_metrics = estimator.evaluate(input_fn=train_input_fn)
eval_metrics = estimator.evaluate(input_fn=eval_input_fn)
print("train metrics: %r"% train_metrics)
print("eval metrics: %r"% eval_metrics)
声明模型输出
>>> estimator = tf.estimator.Estimator(model_fn=model_fn)
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmpzFr6BE
INFO:tensorflow:Using config: {'_save_checkpoints_secs': 600, '_session_config': None, '_keep_checkpoint_max': 5, '_task_type': 'worker', '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x10b3e5950>, '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_service': None, '_num_ps_replicas': 0, '_tf_random_seed': None, '_master': '', '_num_worker_replicas': 1, '_task_id': 0, '_log_step_count_steps': 100, '_model_dir': '/var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmpzFr6BE', '_save_summary_steps': 100}
训练输出
>>> estimator.train(input_fn=input_fn, steps=1000)
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Saving checkpoints for 1 into /var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmpzFr6BE/model.ckpt.
INFO:tensorflow:loss = 238.858812705, step = 1
INFO:tensorflow:global_step/sec: 1083.23
INFO:tensorflow:loss = 0.021875679502, step = 101 (0.092 sec)
INFO:tensorflow:global_step/sec: 1324.24
INFO:tensorflow:loss = 0.00156988131769, step = 201 (0.076 sec)
INFO:tensorflow:global_step/sec: 1243.33
INFO:tensorflow:loss = 0.000146581933847, step = 301 (0.080 sec)
INFO:tensorflow:global_step/sec: 946.093
INFO:tensorflow:loss = 1.70844717209e-05, step = 401 (0.105 sec)
INFO:tensorflow:global_step/sec: 1142.11
INFO:tensorflow:loss = 1.4791159315e-06, step = 501 (0.087 sec)
INFO:tensorflow:global_step/sec: 1191.06
INFO:tensorflow:loss = 1.86484431302e-08, step = 601 (0.084 sec)
INFO:tensorflow:global_step/sec: 1416.25
INFO:tensorflow:loss = 6.86582490973e-09, step = 701 (0.071 sec)
INFO:tensorflow:global_step/sec: 1439.7
INFO:tensorflow:loss = 1.24622057759e-09, step = 801 (0.069 sec)
INFO:tensorflow:global_step/sec: 1402.27
INFO:tensorflow:loss = 4.39855319638e-11, step = 901 (0.071 sec)
INFO:tensorflow:Saving checkpoints for 1000 into /var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmpzFr6BE/model.ckpt.
INFO:tensorflow:Loss for final step: 9.47125781477e-12.
<tensorflow.python.estimator.estimator.Estimator object at 0x10b309e90>
估算输出
>>> train_metrics = estimator.evaluate(input_fn=train_input_fn)
INFO:tensorflow:Starting evaluation at 2017-12-10-03:23:53
INFO:tensorflow:Restoring parameters from /var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmpzFr6BE/model.ckpt-1000
INFO:tensorflow:Finished evaluation at 2017-12-10-03:23:53
INFO:tensorflow:Saving dict for global step 1000: global_step = 1000, loss = 7.43039e-12
>>> eval_metrics = estimator.evaluate(input_fn=eval_input_fn)
INFO:tensorflow:Starting evaluation at 2017-12-10-03:24:09
INFO:tensorflow:Restoring parameters from /var/folders/cs/dwhbchds5x37zkzz2lbnvt_c0000gn/T/tmpzFr6BE/model.ckpt-1000
INFO:tensorflow:Finished evaluation at 2017-12-10-03:24:09
INFO:tensorflow:Saving dict for global step 1000: global_step = 1000, loss = 0.0101005
结果输出
>>> print("train metrics: %r"% train_metrics)
train metrics: {'loss': 7.4303914e-12, 'global_step': 1000}
>>> print("eval metrics: %r"% eval_metrics)
eval metrics: {'loss': 0.010100459, 'global_step': 1000}
欢迎来到这里！
我们正在构建一个小众社区，大家在这里相互信任，以平等 • 自由 • 奔放的价值观进行分享交流。最终，希望大家能够找到与自己志同道合的伙伴，共同成长。
注册 关于