3j8UdZddlmZddlZddlmZddlmZddlm Z ide dd hd e d d hd e d hde dhde dhde ddhde ddhde dhde dhde dhde dhde dhde hd d!e d"d#hd$e d%hd&e d hd'e d(he d)he d*he d+he d,he d-he d.he d/d0he d1he d1he d2d3he d4he d4hd5 Z d6e d7<d dd d(d)d*d+d,d-d d.d d8 Z d9e d:<dd;Z dd<ZeZiddd=d>d?d@ddAdddBdddCdDdEd"ddFdGdHd&dIdJdKdLdMdNdOdPdQdRdSdTidUdVdWdXddYdZd d1d[d4d\d]d^d)d_d d`d+dad-dbd,dcdddedfdgdhdidjdkdldmZd9e dn< ddoZdpZdqe dr<dsZdqe dt<duf ddvZeeZd6e dw<ddxZiZd6e dy<e j3D]=\ZZe exseZe dzeDZej=ee ezee<?eed{|Zd6e d}<dd~Z ddZ!e ddhZ"de d<ddZ# ddZ$y)a$Encoding equivalences and name remapping. This module defines: 1. **Directional supersets** for accuracy evaluation: detecting a superset encoding when the expected encoding is a subset is correct (e.g., detecting UTF-8 when expected is ASCII), but not the reverse. 2. **Bidirectional equivalents**: groups of encodings where detecting any member when another member was expected is considered correct. This includes UTF-16/UTF-32 endian variants (which encode the same text with different byte order) and ISO-2022-JP branch variants (which are compatible extensions of the same base encoding). 3. **Preferred superset mapping** for the ``prefer_superset`` API option: replaces detected ISO/subset encoding names with their Windows/CP superset equivalents that modern software actually uses. 4. **Compatibility names** for the default ``compat_names=True`` mode: maps internal Python codec names to the names chardet 5.x/6.x returned, preserving backward compatibility for callers that compare encoding strings directly. ) annotationsN)Callable) DetectionDictlookup_encodingASCIIzutf-8cp1252zTIS-620 iso8859-11cp874z ISO-8859-11GB2312gb18030GBKBig5 big5hkscscp950 Shift_JIScp932shift_jis_2004zShift-JISX0213zEUC-JP euc_jis_2004z EUC-JISX0213zEUC-KRcp949CP037cp1140z ISO-2022-JP> iso2022_jp_2iso2022_jp_extiso2022_jp_2004z ISO2022-JP-1rrz ISO2022-JP-3rz ISO-8859-1z ISO-8859-2cp1250cp1251cp1256cp1253cp1255cp1254cp1257z utf-16-lez utf-16-bezutf-16z utf-32-lez utf-32-bezutf-32) ISO-8859-5z ISO-8859-6 ISO-8859-7 ISO-8859-8 ISO-8859-9z ISO-8859-13UTF-16z UTF-16-LEz UTF-16-BEUTF-32z UTF-32-LEz UTF-32-BEdict[str, frozenset[str]] SUPERSETS) asciieuc_kr iso8859-1z iso8859-2 iso8859-5z iso8859-6 iso8859-7 iso8859-8 iso8859-9r z iso8859-13tis-620dict[str, str]PREFERRED_SUPERSETcr|jd}t|tr|j|||d<|S)zGReplace the encoding name using *mapping*, modifying *result* in-place.encoding)get isinstancestr)resultmappingencs A/DATA/.local/lib/python3.12/site-packages/chardet/equivalences.py_remap_encodingr>ls4 **Z C#s$[[c2z Mc"t|tS)a7Replace the encoding name with its preferred Windows/CP superset. Modifies the ``"encoding"`` value in *result* in-place and returns *result* for fluent chaining. :param result: A detection result dict containing an ``"encoding"`` key. :returns: The same *result* dict, modified in-place. )r>r4r:s r=apply_preferred_supersetrBts 6#5 66r?cp855IBM855cp866IBM866CP949r,GB18030hzz HZ-GB-2312 iso2022_krz ISO-2022-KRr-r.r#r/r$r0r%r1r&johabJohabzkoi8-rzKOI8-Rz mac-cyrillic MacCyrillicz mac-romanMacRoman SHIFT_JISr2r'r(z utf-8-sigz UTF-8-SIGz Windows-1251z Windows-1252z Windows-1253z Windows-1254z Windows-1255kz1048KZ1048z mac-greekMacGreekz mac-iceland MacIcelandz mac-latin2 MacLatin2z mac-turkish MacTurkish _COMPAT_NAMESc"t|tS)a7Convert internal codec names to chardet 5.x/6.x compatible names. Modifies the ``"encoding"`` value in *result* in-place and returns *result* for fluent chaining. :param result: A detection result dict containing an ``"encoding"`` key. :returns: The same *result* dict, modified in-place. )r>rVrAs r=apply_compat_namesrXs 6= 11r?))rrrtuple[tuple[str, ...], ...]BIDIRECTIONAL_GROUPS))skcs)ukrubgbe)msid)nodasvLANGUAGE_EQUIVALENCESc|SN)xs r=rksr?cfi}|D](}tfd|D}|D] }|||<*|S)zJBuild a lookup: key -> frozenset of all equivalent keys in the same group.c3.K|] }|ywrhri).0n normalizes r= z%_build_group_index..s7A9Q<s) frozenset)groupsrpr:groupnormednames ` r=_build_group_indexrwsC )+F777D&,F9T? # Mr?_LANGUAGE_EQUIVcL||k(rytj|}|duxr||vS)aCheck whether *detected* is an acceptable language for *expected*. Returns ``True`` when *expected* and *detected* are the same ISO 639-1 code, or belong to the same equivalence group in :data:`LANGUAGE_EQUIVALENCES`. :param expected: Expected ISO 639-1 language code. :param detected: Detected ISO 639-1 language code. :returns: ``True`` if the languages are equivalent. TN)rxr7)expecteddetectedrts r=is_language_equivalentr|s48    )E   2U!22r?_NORMALIZED_SUPERSETSc#:K|]}t|xs|ywrhr)rnss r=rqrq sDA*/a/c t|xs|Srhr)ros r=rkrksoa.@.EA.Er?)rp_NORMALIZED_BIDIRc||duS|yt|xs|j}t|xs|j}||k(ry|tvr |t|vry|tvxr |t|vS)aCheck whether *detected* is an acceptable answer for *expected*. Acceptable means: 1. Exact match (after normalization), OR 2. Both belong to the same bidirectional byte-order group, OR 3. *detected* is a known superset of *expected*. :param expected: The expected encoding name, or ``None`` for binary files. :param detected: The detected encoding name, or ``None``. :returns: ``True`` if the detection is acceptable. NFT)rlowerrr})rzr{norm_expnorm_dets r= is_correctrs4x(.8sCd+*?*?*B1ds$$)rrpjoin)textnfkds r=_strip_combiningr5s)   .D 77CdC CCr?)¤€)rrzfrozenset[tuple[str, str]]_EQUIVALENT_SYMBOL_PAIRScR||k(ry||ftvryt|t|k(S)uReturn True if characters *a* and *b* are functionally equivalent. Equivalent means: - Same character, OR - Same base letter after stripping combining marks, OR - An explicitly listed symbol equivalence (e.g. ¤ ↔ €) T)rr)abs r=_chars_equivalentrEs4 Av 1v)) A "21"5 55r?c|||duS|yt|xs|j}t|xs|j}||k(ry |j|}|j|}||k(ryt |t |k7ryt dt||dDS#ttf$rYywxYw)uCheck whether *detected* produces functionally identical text to *expected*. Returns ``True`` when: 1. *detected* is not ``None`` and both encoding names normalize to the same codec, OR 2. Decoding *data* with both encodings yields identical strings, OR 3. Every differing character pair is functionally equivalent: same base letter after stripping combining marks, or an explicitly listed symbol equivalence (e.g. ¤ ↔ €). Returns ``False`` if *detected* is ``None``, either encoding is unknown, or either encoding cannot decode *data*. :param data: The raw byte data that was detected. :param expected: The expected encoding name, or ``None`` for binary files. :param detected: The detected encoding name, or ``None``. :returns: ``True`` if decoding with *detected* yields functionally identical text to decoding with *expected*. NFTc3:K|]\}}t||ywrh)r)rnrrs r=rqz*is_equivalent_detection..sX3W41a A&3Wr)strict)rrdecodeUnicodeDecodeError LookupErrorlenallzip)datarzr{rrtext_exptext_dets r=is_equivalent_detectionrUs.4x(rBapply_legacy_renamerVrXrZrfrwrxr|r}items_subset _supersets_key_normedr7rrrrrrrir?r=rs[0#$*,+( Y* ++( y,01+(9gY'+( i $ +(  9i[ ! +(  I{G, - +(G%567+(i!1 23+( i()+(I~./+( i "+( Yz "+(.9RS/+(0I~/?@A1+(2I0123+(6)XJ'7+(8)XJ'9+(:XJ'XJ'XJ'XJ'XJ'hZ(k23H:&H:&k23H:&H:&U+( $+f &N   7  7 7/%!%! X%! X %!  W %! H %! h%!y%! ,%!M%!-%!%!%!%!%! !%!" W#%!$ h%%!&M'%!()%!*k+%!,y-%!. h/%!0 h1%!23%!4 n5%!6 n7%!8 n9%!: n;%!< n=%!@ hA%!BC%!D