网页设计报价标准_潞城网络营销怎么做_网站seo优化分析_googleplay官网

综合示例：文本分类Transformer

要将词序考虑在内，你只需将Embedding层替换为位置感知的PositionEmbedding层，如代码清单11-25所示。

代码清单11-25　将Transformer编码器与位置嵌入相结合

vocab_size = 20000
sequence_length = 600
embed_dim = 256
num_heads = 2
dense_dim = 32inputs = keras.Input(shape=(None,), dtype="int64")
x = PositionalEmbedding(sequence_length, vocab_size, embed_dim)(inputs)  ←----注意这行代码！
x = TransformerEncoder(embed_dim, dense_dim, num_heads)(x)
x = layers.GlobalMaxPooling1D()(x)
x = layers.Dropout(0.5)(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs