I don't know why but when I use the search_tweets2 function from rtweet package, it behaves differently every time I try to retrieve tweets from the database. Sometimes it retrieves the number of rows I request, but mostly it crashes after some progress. It doesn't matter what search words I am using, my impression is that repeated requests with the same search words would end up with the error. I also reloaded the R environment countless times, even changed my IP-Address - this does not have any influence on the behavior of the function. Here's my simple code:
pl_hrb <- search_tweets2(+ c(" herbal OR Herbal OR yog* OR Yog* "),+ n = 50000, retryonratelimit = TRUE, lang = "en"+ )and here are the outputs:
Crash 1:
Downloading [>----------------------------------------] 2%> Crash 2:
Downloading [======>----------------------------------] 24%> Almost done:
Downloading [=========================================] 100%retry on rate limit...waiting about 5 minutes...Downloading [=========================================] 100%retry on rate limit...waiting about 12 minutes...Downloading [===================>---------------------] 50%>Success:
Downloading [=========================================] 100%retry on rate limit...waiting about 10 minutes...Downloading [=========================================] 100%retry on rate limit...waiting about 15 minutes...Downloading [=========================================] 100%>What am I doing wrong? Why is this discrepancy?Thank you in advance.