问题描述
从DBeaver中导出了部分业务数据的 insert sql,明明在开发、测试环境都可以一把执行通过,却在预发环境执行前的语法检查失败了,提示有SQL语法错误。
这条SQL长这样,default_sql是要在odps上执行的sql语句,提前配置好放数据库里,方便后续读取。
INSERT INTO t_profile_attribute (`attribute`, default_sql) VALUES('province', 'WITH t AS (
SELECT b.analysis_dim1
,b.analysis_dim2
,a.uid
FROM (
SELECT uid
FROM {kun_table}
WHERE ds = MAX_PT("{kun_table}")
) a
LEFT JOIN (
SELECT uid, COALESCE(sale_province, ''未知'') AS analysis_dim1, analysis_dim2
FROM usr_user_group_insight_df
LATERAL VIEW OUTER EXPLODE(SPLIT({split_field}, '','')) t1 AS analysis_dim2
WHERE ds = MAX_PT("usr_user_group_insight_df")
) b ON a.uid = b.uid
)
SELECT
analysis_dim2,
analysis_dim1,
COUNT(1) AS cnt
FROM t
WHERE analysis_dim1 NOT IN ('''', ''未知'') and analysis_dim2 NOT IN ('''', ''未知'')
GROUP BY analysis_dim2, analysis_dim1
ORDER BY cnt DESC;');
原因分析:
MySQL数据库使用了不同的转义规则来处理特殊字符。
对于换行符,MySQL将其视为一个特殊字符,需要进行转义处理才能正确保存。
如果我们没有正确处理换行符,MySQL将无法正确解析文本中的换行符,就可能会导致数据保存失败。
解决方案:
替换特殊符号,需要将\n
替换成\\n
,SQL才能正常执行。
我是直接在IDEA文件中,使用ctrl+r, 将\n
替换成\\n
。
原DBeaver导出SQL:
INSERT INTO t_profile_attribute (`attribute`, default_sql) VALUES('province', 'WITH t AS (
SELECT b.analysis_dim1
,b.analysis_dim2
,a.uid
FROM (
SELECT uid
FROM {kun_table}
WHERE ds = MAX_PT("{kun_table}")
) a
LEFT JOIN (
SELECT uid, COALESCE(sale_province, ''未知'') AS analysis_dim1, analysis_dim2
FROM usr_user_group_insight_df
LATERAL VIEW OUTER EXPLODE(SPLIT({split_field}, '','')) t1 AS analysis_dim2
WHERE ds = MAX_PT("usr_user_group_insight_df")
) b ON a.uid = b.uid
)
SELECT
analysis_dim2,
analysis_dim1,
COUNT(1) AS cnt
FROM t
WHERE analysis_dim1 NOT IN ('''', ''未知'') and analysis_dim2 NOT IN ('''', ''未知'')
GROUP BY analysis_dim2, analysis_dim1
ORDER BY cnt DESC;');
替换完成后的SQL:
INSERT INTO t_profile_attribute ( `attribute`, default_sql, ) VALUES
('province', 'WITH t AS (\nSELECT b.analysis_dim1\n,b.analysis_dim2\n,a.uid\nFROM (\nSELECT uid\nFROM {kun_table}\nWHERE ds = MAX_PT("{kun_table}")\n) a\nLEFT JOIN (\nSELECT uid, COALESCE(sale_province, ''未知'') AS analysis_dim1, analysis_dim2\nFROM user_group_insight_df\nLATERAL VIEW OUTER EXPLODE(SPLIT({split_field}, '','')) t1 AS analysis_dim2\nWHERE ds = MAX_PT("user_group_insight_df")\n) b ON a.uid = b.uid\n)\nSELECT\nanalysis_dim2,\nanalysis_dim1,\nCOUNT(1) AS cnt\nFROM t\nWHERE analysis_dim1 NOT IN ('''', ''未知'') and analysis_dim2 NOT IN ('''', ''未知'')\nGROUP BY analysis_dim2, analysis_dim1\nORDER BY cnt DESC;')