• TensorFlow 集成入门
    • 编译 MLeap-TensorFlow 模块
    • 使用 MLeap-TensorFlow

    TensorFlow 集成入门

    MLeap Tensorflow 集成允许用户将 TensorFlow Graph 当做 Transformer 集成到 ML Pipeline 中。在未来我们会提升对 TensorFlow 的兼容性。目前来说，TensorFlow 与 MLeap 的集成还是一个实验性功能，我们仍在进一步稳定这个特性。

    编译 MLeap-TensorFlow 模块

    MLeap TensorFlow 模块未被托管在 Maven Central 上，用户必须借助 TensorFlow 提供的 JNI（Java Native Interface）支持，编译源码获得。参考相关教程从源码编译 TensorFlow 模块。

    使用 MLeap-TensorFlow

    编译工作就绪之后，你就能轻松将 TensorFlow 集成到 MLeap Pipeline 中。

    首先，添加 MLeap-TensorFlow 作为项目依赖。

    1. libraryDependencies += "ml.combust.mleap" %% "mleap-tensorflow" % "0.13.0"

    接下来就能在代码中使用 Tensor Graph。让我们构建一个包含两个 Tensor 的简单 Graph。

    1. import ml.combust.mleap.core.types._
    2. import ml.combust.mleap.runtime.frame.{DefaultLeapFrame, Row}
    3. import ml.combust.mleap.tensor.Tensor
    4. import ml.combust.mleap.tensorflow.{TensorflowModel, TensorflowTransformer}
    5. import org.tensorflow
    7. // Initialize our Tensorflow demo graph
    8. val graph = new tensorflow.Graph
    10. // Build placeholders for our input values
    11. val inputA = graph.opBuilder("Placeholder", "InputA").
    12.   setAttr("dtype", tensorflow.DataType.FLOAT).
    13.   build()
    14. val inputB = graph.opBuilder("Placeholder", "InputB").
    15.   setAttr("dtype", tensorflow.DataType.FLOAT).
    16.   build()
    18. // Multiply the two placeholders and put the result in
    19. // The "MyResult" tensor
    20. graph.opBuilder("Mul", "MyResult").
    21.   setAttr("T", tensorflow.DataType.FLOAT).
    22.   addInput(inputA.output(0)).
    23.   addInput(inputB.output(0)).
    24.   build()
    26. // Build the MLeap model wrapper around the Tensorflow graph
    27. val model = TensorflowModel(graph,
    28.   // Must specify inputs and input types for converting to TF tensors
    29.   inputs = Seq(("InputA", TensorType.Float()), ("InputB", TensorType.Float())),
    30.   // Likewise, specify the output values so we can convert back to MLeap
    31.   // Types properly
    32.   outputs = Seq(("MyResult", TensorType.Float())))
    34. // Connect our Leap Frame values to the Tensorflow graph
    35. // Inputs and outputs
    36. val shape = NodeShape().
    37.   // Column "input_a" gets sent to the TF graph as the input "InputA"
    38.   withInput("InputA", "input_a").
    39.   // Column "input_b" gets sent to the TF graph as the input "InputB"
    40.   withInput("InputB", "input_b").
    41.   // TF graph output "MyResult" gets placed in the leap frame as col
    42.   // "my_result"
    43.   withOutput("MyResult", "my_result")
    45. // Create the MLeap transformer that executes the TF model against
    46. // A leap frame
    47. val transformer = TensorflowTransformer(shape = shape, model = model)
    49. // Create a sample leap frame to transform with the Tensorflow graph
    50. val schema = StructType(StructField("input_a", ScalarType.Float), StructField("input_b", ScalarType.Float)).get
    51. val dataset = Seq(Row(5.6f, 7.9f),
    52.   Row(3.4f, 6.7f),
    53.   Row(1.2f, 9.7f))
    54. val frame = DefaultLeapFrame(schema, dataset)
    56. // Transform the leap frame and make sure it behaves as expected
    57. val data = transformer.transform(frame).get.dataset
    58. assert(data(0)(2).asInstanceOf[Tensor[Float]].get(0).get == 5.6f * 7.9f)
    59. assert(data(1)(2).asInstanceOf[Tensor[Float]].get(0).get == 3.4f * 6.7f)
    60. assert(data(2)(2).asInstanceOf[Tensor[Float]].get(0).get == 1.2f * 9.7f)
    62. // Cleanup the transformer
    63. // This closes the TF session and graph resources
    64. transformer.close()

    更多关于 TensorFlow 集成如何运作的细节：

    1. 数据集成与转换的相关细节参见本章节。
    2. 序列化 TensorFlow Graph 为 MLeap Bundle 的相关细节参见本章节。