o z,hg@s\dZddlZddlmZddlmZddlmZmZddl m Z GdddZ d d d Z dS) z SQL LexerN)Lock) TextIOBase)tokenskeywords)consumec@sXeZdZdZdZeZeddZddZ ddZ d d Z d d Z d dZ dddZdS)LexerzrThe Lexer supports configurable syntax. To add support for additional keywords, use the `add_keywords` method.NcCs^|j!|jdur||_|jWd|jSWd|jS1s'wY|jS)zRReturns the lexer instance used internally by the sqlparse core functions.N)_lock_default_instancedefault_initialization)clsr D/var/www/html/rh/venv/lib/python3.10/site-packages/sqlparse/lexer.pyget_default_instance0s    zLexer.get_default_instancecCs||tj|tj|tj|tj|tj|tj |tj |tj |tj |tj dS)zlInitialize the lexer with default dictionaries. Useful if you need to revert custom syntax settings.N)clear set_SQL_REGEXr SQL_REGEX add_keywordsKEYWORDS_COMMONKEYWORDS_ORACLEKEYWORDS_MYSQLKEYWORDS_PLPGSQL KEYWORDS_HQLKEYWORDS_MSACCESSKEYWORDS_SNOWFLAKEKEYWORDS_BIGQUERYKEYWORDSselfr r r r :s         zLexer.default_initializationcCsg|_g|_dS)zClear all syntax configurations. Useful if you want to load a reduced set of syntax configurations. After this call, regexps and keyword dictionaries need to be loaded to make the lexer functional again.N) _SQL_REGEX _keywordsrr r r rIs z Lexer.clearcs$tjtjBfdd|D|_dS)z.Set the list of regex that will parse the SQL.cs"g|] \}}t|j|fqSr )recompilematch).0rxttFLAGSr r Tsz'Lexer.set_SQL_REGEX..N)r IGNORECASEUNICODEr)rrr r&r rQs   zLexer.set_SQL_REGEXcCs|j|dS)zhAdd keyword dictionaries. Keywords are looked up in the same order that dictionaries were added.N)rappend)rrr r r rYszLexer.add_keywordscCs6|}|jD]}||vr|||fSqtj|fS)zChecks for a keyword. If the given value is in one of the KEYWORDS_* dictionary it's considered a keyword. Otherwise, tokens.Name is returned. )upperrrName)rvaluevalkwdictr r r is_keyword^s   zLexer.is_keywordc cs t|tr |}t|trn,t|tr3|r||}nz|d}Wnty2|d}Yn wtdt |t |}|D]A\}}|j D]3\}}|||}|sUqIt|t j rc||fVn |tjurp||Vt|||dnt j|fVqBdS)a Return an iterable of (tokentype, value) pairs generated from `text`. If `unfiltered` is set to `True`, the filtering mechanism is bypassed even if filters are defined. Also preprocess the text, i.e. expand tabs and strip it if wanted and applies registered filters. Split ``text`` into (tokentype, text) pairs. ``stack`` is the initial stack (default: ``['root']``) zutf-8zunicode-escapez+Expected text or file-like object, got {!r}N) isinstancerreadstrbytesdecodeUnicodeDecodeError TypeErrorformattype enumeraterr _TokenTypegrouprPROCESS_AS_KEYWORDr1rendError) rtextencodingiterableposcharrexmatchactionmr r r get_tokensks>           zLexer.get_tokensN)__name__ __module__ __qualname____doc__r rr classmethodrr rrrr1rJr r r r rs   rcCst||S)zTokenize sql. Tokenize *sql* using the :class:`Lexer` and return a 2-tuple stream of ``(token type, value)`` items. )rrrJ)sqlrCr r r tokenizesrRrK) rOr threadingriorsqlparserrsqlparse.utilsrrrRr r r r s