[i2ddlZddlZddlZddlZddlZej ddkDrddlmZddlm Z m Z ddl m Z m Z mZnddlmZddlm Z ddlm Z m Z ddl m Z mZ ddlmZd Zgd Zd Zd ZdZdZdZdZej:dZesej:dZesdZeej>jAedZ! e!jEdZ$ ej>jKej>jMe'dZ( ej>jAe(dZ)ddl*Z*e*jVe)dZ, e,j[Dcgc]}|j]c}Z/e,ja[, dZ1dZ2d dZ3dZ4 d!dZ5dZ6y#e$r ddlmZd ZYBwxYw#e#$rYwxYwcc}w#e,ja[,wxYw#e#$roej>jAe(dZ)e+e)5Z,e,j[Dcgc]}|j]ncc}wc}Z/dddn #1swYnxYwYwxYw#e#$re$gZ/YwxYw)"N) LWPCookieJar)Requesturlopen) quote_plusurlparseparse_qs)r)rr ) BeautifulSoupTF)searchluckyget_random_user_agentget_tbszhttps://www.google.%(tld)s/zuhttps://www.google.%(tld)s/search?hl=%(lang)s&q=%(query)s&btnG=Google+Search&tbs=%(tbs)s&safe=%(safe)s&cr=%(country)szrhttps://www.google.%(tld)s/search?hl=%(lang)s&q=%(query)s&start=%(start)d&tbs=%(tbs)s&safe=%(safe)s&cr=%(country)szhttps://www.google.%(tld)s/search?hl=%(lang)s&q=%(query)s&num=%(num)d&btnG=Google+Search&tbs=%(tbs)s&safe=%(safe)s&cr=%(country)sz~https://www.google.%(tld)s/search?hl=%(lang)s&q=%(query)s&num=%(num)d&start=%(start)d&tbs=%(tbs)s&safe=%(safe)s&cr=%(country)s)hlqnumbtnGstarttbssafecrHOMEUSERHOME.z.google-cookiez2Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0)zuser_agents.txt.gzrbzuser_agents.txtc4tjtS)za Get a random user agent string. :rtype: str :return: Random user agent string. )randomchoiceuser_agents_listN/mnt/data/GridWatch/venv/lib/python3.12/site-packages/googlesearch/__init__.pyr r {s ==) **r c`|jd}|jd}dtzS)z Helper function to format the tbs parameter. :param datetime.date from_date: Python date object. :param datetime.date to_date: Python date object. :rtype: str :return: Dates encoded in tbs format. z%m/%d/%Yz-cdr:1,cd_min:%(from_date)s,cd_max:%(to_date)s)strftimevars) from_dateto_dates r!rrs1"":.Iz*G :TV CCr c|t}t|}|jd|tj ||r t |}n!t j}t ||}tj|||j}|j tj|S#t$rY|SwxYw)aL Request the given URL and return the response page, using the cookie jar. :param str url: URL to retrieve. :param str user_agent: User agent for the HTTP requests. Use None for the default. :param bool verify_ssl: Verify the SSL certificate to prevent traffic interception attacks. Defaults to True. :rtype: str :return: Web page retrieved for the given URL. :raises IOError: An exception is raised on error. :raises urllib2.URLError: An exception is raised on error. :raises urllib2.HTTPError: An exception is raised on error. z User-Agent)context) USER_AGENTr add_header cookie_jaradd_cookie_headerrssl_create_unverified_contextextract_cookiesreadclosesave Exception)url user_agent verify_sslrequestresponser(htmls r!get_pager:s" clG |Z0  )7#0027G4x1 ==?D NN  K   K s!B77 CCc |jdr't|d}t|jdd}t|d}|jrd|jvr|Syy#t $rYywxYw)Nz/url?httprrgoogle) startswithrr querynetlocr3)linkos r! filter_resultrCsy  ??7 #v&AAGG$S)!,D T6 " 880K18    sAA$$ A0/A0c #~Kt} d}t|}| si} tD] }|| jvst d|t t tz| | |r)|dk(rttz}n:ttz}n(|dk(rttz}nttz}|r||kr+|}| jD]&\}}t|}t|}|d|d|z}(tj|t || | }tr t!|d}n t!|} |j#dj%d }|D]J} |d }t-|}|st/|}|| vr'| j1|||d z }|sD||k\sJy ||k(ry ||z }|dk(rttz}nttz}|s#||kr*y y #t&$r8|j#d }|r|j)|j%d }YwxYw#t*$rYwxYww)a Search the given query string using Google. :param str query: Query string. Must NOT be url-encoded. :param str tld: Top level domain. :param str lang: Language. :param str tbs: Time limits (i.e "qdr:h" => last hour, "qdr:d" => last 24 hours, "qdr:m" => last month). :param str safe: Safe search. :param int num: Number of results per page. :param int start: First result to retrieve. :param int stop: Last result to retrieve. Use None to keep searching forever. :param float pause: Lapse to wait between HTTP requests. A lapse too long will make the search slow, but a lapse too short may cause Google to block your IP. Your mileage may vary! :param str country: Country or region to focus the search on. Similar to changing the TLD, but does not yield exactly the same results. Only Google knows why... :param dict extra_params: A dictionary of extra HTTP GET parameters, which must be URL encoded. For example if you don't want Google to filter similar results you can set the extra_params to {'filter': '0'} which will append '&filter=0' to every query. :param str user_agent: User agent for the HTTP requests. Use None for the default. :param bool verify_ssl: Verify the SSL certificate to prevent traffic interception attacks. Defaults to True. :rtype: generator of str :return: Generator (iterator) that yields found URLs. If the stop parameter is None the iterator will loop forever. rzQGET parameter "%s" is overlapping with the built-in GET parameter &=z html.parserr )idagbarhrefN)setrurl_parameterskeys ValueErrorr:url_homer$ url_next_pageurl_next_page_num url_searchurl_search_numitemstimesleepis_bs4r findfindAllAttributeErrorclearKeyErrorrChashadd)r?tldlangrrrrstoppausecountry extra_paramsr5r6hashescount builtin_paramr4 last_countkvr9soupanchorsrJrIrAhs r!r r seJUF E u E  ( L--/ /,  X  J7 "9$&(C#df,C "9tv%C 46)Cedl !&&( ,DAq1 A1 AQ*+C , 5Z4  }5D &D (ii8i,44S9G A y !&DT AF{ JJqMJ QJE 5 <      "9$&(C#df,CWedl: (999'D ll3'G  (  sk7H=C?H=:!G*H=!H.&:H=!H='8H=!H=(H=*>H+(H=*H++H=. H:7H=9H::H=c*tt|i|S)z Shortcut to single-item search. Same arguments as the main search function, but the return value changes. :rtype: str :return: URL found by Google. )nextr )argskwargss r!r r os '' ((r )NT) comen0offrErNg@NNT)7osrsysrWr- version_infohttp.cookiejarrurllib.requestrr urllib.parserrr cookieliburlliburllib2bs4r rY ImportError__all__rQrTrRrUrSrNgetenv home_folderpathjoinr+loadr3r)abspathsplit__file__install_folderuser_agents_filegzipopenfp readlinesstriprr1r rr:rCr r )_s0r!rsY< A+/;;&!(+! F  ) ! "3< bii "))J'K  "'',,{4DE F  OO B $WW__RWW]]8%Ba B  HHJ+ D""N *LM:>'+W-x )} + FV  $ C HHJ C77<<8IJ " # Cr35<<>Ba BB  C C CC$"|$sFF+7I 3GF;F6*F;-G>I F('F(+F32F36F;;GG)I:H6 H%$ H6- I6H? ;IIIIII