Pages

Sunday, May 7, 2023

Remove non ASCII character in bash and sql

Remove non ASCII character in bash

remove_non_ascii.sh
#!/bin/bash
WORK_DIR=$1
FILE=$2
RUN_DATE=`date "+%Y%m%d"_"%H%M"`
input=${WORK_DIR}/${FILE}
output=${WORK_DIR}/${FILE}_${RUN_DATE}
while IFS= read -r line
do
  echo "$line" | tr -cd [:print:] >> ${output}
  echo >> ${output}
done < "$input"
mv ${output} ${input}

Usage:
./remove_non_ascii.sh "/some/path" "some_file.txt"


Find non ASCII character in SQL
SELECT * FROM MY_TABLE 
 WHERE column_name != ASCIISTR(column_name);

For example:
SELECT 'GSM_COUNTRIES' AS table_name, 'country_name' as column_name, country_name as column_value  
  FROM GSM_COUNTRIES my_table   
 WHERE country_name != ASCIISTR(country_name);

To generate the code:

SELECT 'SELECT '||''''||USER_TAB_COLUMNS.table_name||''' AS table_name, '||''''||USER_TAB_COLUMNS.column_name||''' as column_name, '||USER_TAB_COLUMNS.column_name||' as column_value  FROM '||USER_TAB_COLUMNS.table_name||' my_table   WHERE '||USER_TAB_COLUMNS.column_name||' != ASCIISTR('||USER_TAB_COLUMNS.column_name||');'
  FROM USER_TAB_COLUMNS 
 WHERE table_name = 'GSM_COUNTRIES'  AND data_type = 'VARCHAR2'
 ORDER BY column_name;


No comments:

Post a Comment