Avoid 'Rate limit exceeded' error on 'ls' command

When listing friends or followers, the Twitter API sends them back to
you by batches of 20 users. If there are more, you then have to make
another request.

With more than 300 friends (or followers), this leads you to make more
than 15 calls. Unfortunately, the rate limit for this call is set at 15
calls per 15 mn.

This patch makes two modifications.

First, the users are printed out as they are received, and not once they
are all received.
Second, once you made 15 API requests, the program will wait for 16mn
before pursuing, in order to avoid the 'Rate limit exceeded' error.

This is not ideal (the stream is effectiveley in pause for 16mn...), but
still better that the old way, when, with 400 friends, all I could get
from 'ls fr' was 'Rate limit exceeded'. :)

diff --git a/rainbowstream/rainbow.py b/rainbowstream/rainbow.py
index 1b5ee1d98ff8c2cff2e1700b59a7fd96a0aaff90..e0668c32522506195414415a551030659f1a56fb 100644 (file)
--- a/rainbowstream/rainbow.py
+++ b/rainbowstream/rainbow.py
@@ -903,24 +903,40 @@ def ls():
     d = {'fl': 'followers', 'fr': 'friends'}
     next_cursor = -1
     rel = {}
     d = {'fl': 'followers', 'fr': 'friends'}
     next_cursor = -1
     rel = {}

+
+    printNicely('All ' + d[target] + ':')
+

     # Cursor loop
     # Cursor loop

+    number_of_users = 0

     while next_cursor != 0:
     while next_cursor != 0:

+

         list = getattr(t, d[target]).list(
             screen_name=name,
             cursor=next_cursor,
             skip_status=True,
             include_entities=False,
         )
         list = getattr(t, d[target]).list(
             screen_name=name,
             cursor=next_cursor,
             skip_status=True,
             include_entities=False,
         )

+

         for u in list['users']:
         for u in list['users']:

-            rel[u['name']] = '@' + u['screen_name']
+
+            number_of_users += 1
+
+            # Print out result
+            printNicely(   '  '                                               \
+                         + cycle_color( u['name'] )                           \
+                         + color_func(c['TWEET']['nick'])(    ' @'            \
+                                                           + u['screen_name'] \
+                                                           + ' ' ) )
+

         next_cursor = list['next_cursor']
         next_cursor = list['next_cursor']

-    # Print out result
-    printNicely('All: ' + str(len(rel)) + ' ' + d[target] + '.')
-    for name in rel:
-        user = '  ' + cycle_color(name)
-        user += color_func(c['TWEET']['nick'])(' ' + rel[name] + ' ')
-        printNicely(user)

 
 

+        # 300 users means 15 calls to the related API. The rate limit is 15
+        # calls per 15mn periods (see Twitter documentation).
+        if ( number_of_users % 300 == 0 ):
+            printNicely( '(waiting 16mn for rate limits reasons...)' )
+            time.sleep(16*60)
+
+    printNicely('All: ' + str(number_of_users) + ' ' + d[target] + '.')

 
 def follow():
     """
 
 def follow():
     """