Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions tn/chinese/data/cardinal/phone_after.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
电话
電話
打电话
打電話
打个电话
打個電話
报警
報警
火警
急救
救护车
救護車
热线
熱線
查号
查號
交警
指挥中心
指揮中心
10 changes: 10 additions & 0 deletions tn/chinese/data/cardinal/phone_before.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
拨打
撥打
拨
撥
打
打给
打給
致电
致電
呼叫
12 changes: 10 additions & 2 deletions tn/chinese/rules/cardinal.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,12 +79,20 @@ def build_tagger(self):
cardinal |= digits.plus + (delete("-") + digits.plus)**2
# xxx-xxxxxxxx
cardinal |= digits**3 + delete("-") + digits**8
# well-known short phone numbers (110, 12306, etc.) and 11-digit mobile
# well-known 5-digit service numbers (12306, etc.) and 11-digit mobile
phone_digits = digits @ self.build_rule(cross("一", "幺"))
phone = string_file(get_abs_path("chinese/data/cardinal/phone.tsv"))
known_phones = string_file(get_abs_path("chinese/data/cardinal/phone.tsv"))
phone = self.DIGIT**5 @ known_phones
phone |= phone_digits**11
phone |= accep("尾号") + (accep("是") | accep("为")).ques + phone_digits**4
cardinal |= add_weight(phone, -1.0)
# 110, 120, etc. are also ordinary quantities (120题, 110公尺), so read them
# as phone numbers only next to a word that says so (拨打120, 120急救)
emergency = self.DIGIT**3 @ known_phones
rmspace = delete(" ").ques
before = string_file(get_abs_path("chinese/data/cardinal/phone_before.tsv"))
after = string_file(get_abs_path("chinese/data/cardinal/phone_after.tsv"))
cardinal |= before + rmspace + emergency | emergency + rmspace + after

self.cardinal = cardinal.optimize()
self.tagger = self.add_tokens(self.tag_field("value", self.cardinal))
Expand Down
4 changes: 3 additions & 1 deletion tn/chinese/test/data/cardinal.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
110 => 幺幺零
拨打110 => 拨打幺幺零
拨打 119 => 拨打幺幺九
120急救 => 幺二零急救
2% => 百分之二
127.0.0.1 => 一二七点零点零点一
010-64035547 => 零一零六四零三五五四七
Expand Down
8 changes: 8 additions & 0 deletions tn/chinese/test/data/normalizer.txt
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,11 @@ B2B => B to B
包含3000余件 => 包含三千余件
查处450余名 => 查处四百五十余名
查处450余名 => 查处四百五十余名
共120题 => 共一百二十题
跑了110公尺 => 跑了一百一十公尺
总分是120 => 总分是一百二十
快打110报警 => 快打幺幺零报警
有事打120 => 有事打幺二零
撥打119 => 拨打幺幺九
他打了120分 => 他打了一百二十分
给110打个电话 => 给幺幺零打个电话