Fun with Google Voice Transcripts

I love the Google Voice voice mail transcripts, but sometimes, they are only useful for their entertainment value.

Here’s one of my favorite transcripts:

I think I got the only thing I cannot hold. Thanking towns elite. Thank god speed that. So it looks like we’re on for Chris and that’s what we’re going to wrap self See You Later. Bye Bye.

Here’s what you actually said:

I think I got the only duck in the whole stinking town, but at least I got me a duck, so it looks like we are on for crispy duck as well as lettuce wrap, so…see ya later, bye bye.

Action Required: Preserve Progress Road Bike Route

Spokane Valley cyclists, act now to preserve an important bike route. We are in jeopardy of losing the signal light at Progress and Sprague. Email inote@spokanevalley.org">Inga Note, Senior Traffic Engineer, or call her at (509) 720-5011. Let Inga know you use this route and it is important for cyclists. Without the traffic signal, we lose an important north-south route along the Sullivan Road corridor. Progress is designated as a preferred bike route in the new Bike and Pedestrian Master Program. Don’t let it disappear.

For background, here’s an email I received from Inga:

Marc,

We recently received funding for the reconstruction of Sprague Avenue from Evergreen to Sullivan. This will require us to make some significant upgrades to the signals at Adams and Progress. We have taken several traffic counts at the intersection and the volumes are low enough that the signal no longer meets signal warrants as defined by the Manual on Uniform Traffic Control Devices. The purpose of the MUTCD warrants is to ensure that traffic signals are installed in a consistent manner throughout the country. Some of the issues we are considering are outlined below:

  • The signal provides a protected crossing for pedestrians and cyclists. Progress was recently designated as a Bicycle Route on the City’s Bicycle and Pedestrian Plan.
  • The signal allows for pedestrian and cyclist access to transit stops and businesses.
  • It allows for through traffic on Progress to easily cross Sprague.
  • It provides a protected turn movement for traffic entering Sprague or turning onto Progress.
  • The spacing of the Progress and Adams signals is only ¼ mile. ½ mile spacing is the standard for most Sprague signals and allows for better signal coordination and traffic flow.
  • Removal of the signal would result in lower volumes on Progress, which is classified as a local street. Through traffic would likely shift to Adams, which is classified as a minor arterial.
  • There would be impacts to the nearby schools. Likely an increase in traffic near Adams Elementary and a decrease in traffic near Progress Elementary.
  • Keeping unwarranted signals in place has ongoing costs for maintenance and electricity.
  • During the peak hours the signal operates on a fixed cycle in order to maintain traffic flow on Sprague. This sometimes results in longer wait times for traffic making a left turn off Sprague or Progress. During non-peak travel times the signal will stop larger traffic volumes on Sprague to allow left turns or side street traffic to go.

As part of the public notification we installed “traffic signal under study for removal” signs on all four approaches to the signal. I have already received several calls (both in favor and against) from concerned citizens. It is particularly difficult for us to gauge the non-motorized use of the signal, so the callers are providing valuable information in that regard. At this time we have not made a decision on the removal and want to hear any comments people have regarding safety, pedestrian and transit access, business impacts, etc.

So far I’ve mostly heard from pedestrians, but if you have knowledge of the cyclist use of this intersection that you could share with me, that would be very helpful.

Thanks, Inga Note Senior Traffic Engineer

Mapping Bike Routes with GPS

The City of Spokane Valley has been working on its Bike and Pedestrian Master Program to identify and fill the needs of non-motorized road users.

Two community workshops were held last year to help identify needs, existing routes, and trouble spots. The information gathered was certainly good and useful, but it came from a relatively small group of citizens and was quite subjective.

I’ve been thinking about how communities can gather more objective data.

Here’s an idea: attach inexpensive GPS receivers to bikes. Collect data for 4-6 weeks and see where bike commuters actually go. If you can get a large enough sample, you should be able to tell which routes and destinations are popular. You should be able to identify obstacles and trouble spots by the routes cyclists take to avoid them. The GPS data would give you highly objective information including both time and location.

Such objective data could help planners make the best use of limited resources.

I wonder if Google Maps or another high tech company would be willing to partner with cities to help them gather this information? The data gathered might help them, as well, to provide better information to their users.

Do you know anyone who might be interested in pursuing this? Is anyone already doing it? Let me know.

Hijacking LWP::UserAgent - just don't hang yourself

Perl gives you enough rope to hang yourself, which can be useful when you feel like hanging someone else.

I was using some library code littered with calls like this:

my $response = LWP::UserAgent->new->get($uri);

The default arguments to LWP::UserAgent weren’t sufficient for my needs, and changing the library code wasn’t a reasonable option—at least not in the short term. So I used some perl rope to make a noose:

use LWP::UserAgent;
{
    my $user_agent = LWP::UserAgent->new(%my_custom_args);
    no warnings 'redefine';
    *LWP::UserAgent::new = sub { $user_agent };
}

There are all sorts of reasons you might not want to use the default arguments. In fact, you may not want to use LWP::UserAgent. Perhaps you want to use LWP::UserAgent::POE or AnyEvent::HTTP::LWP::UserAgent or the fast and furious Furl instead.

Hijacking is a hanging offense, so be careful not to get your own neck in the noose.

Twitter API Change Breaks Existing Code

Your Twitter Apps may break

On 25-Mar-2011, Twitter announced changes to the API. If your code currently uses the friends_ids or followers_ids methods, without the cursor parameter, it will soon break!

Twitter intends to start forcing the use of cursored calls. If you do not provide a cursor parameter, Twitter will add one for you, with value -1. This will change the return value! Instead of getting an array reference, you will get a hash reference, with the array of returned IDs nested within it.

Let’s pick on @miyagawa. He as a lot of followers.

my $r = $nt->followers_ids('miyagawa');

The call above currently returns 9,191 IDs:

[ 151398039, 272906704, 197990177, 9431032, 271536579... ]

When Twitter begins forcing cursored calls, that same call will return:

{
   ids => [
       151398039, 272906704, 197990177, 9431032, 271536579...
   ],
   next_cursor         => 1317952939226585828,
   next_cursor_str     => 1317952939226585828,
   previous_cursor     => 0,
   previous_cursor_str => 0,
}

where $r->{ids} contains @miyagawa’s most recent 5,000 followers.

The Net::Twitter documentation includes information on how to use the cursor parameter.

To help with the transition, I created a new trait for Net::Twitter which will transparently handled cursored calls.

AutoCursor

When applied, the AutoCursor trait will first attempt a friends_ids or followers_ids call without a cursor parameter (unless you supply one). If Twitter forces a cursored call by adding the cursor parameter, AutoCursor will automatically call the underlying API method in a loop for up to 16 calls and return a reference to an array with the combined results.

This limits the change required to your code to just the Net::Twitter->new call and handles users with up to 80,000 friends or followers transparently.

The AutoCursor trait is parameterized, so you can customize its behavior:

$nt = Net::Twitter->new(
    traits => [
        qw/API::REST RetryOnError OAuth/,
        AutoCursor => { max_calls => 40 },
    ],
    # additional new arguments...
);

With this max_calls parameter, followers_ids will make up to 40 underlying API calls returning up to 200,000 IDs.

When using AutoCursor if you supply a cursor parameter to the call, it will use that parameter for the first call, rather than doing an initial cursor-less call.

You can also use AutoCursor for other calls that accept a cursor parameter. For example:

$nt = Net::Twitter->new(
    traits => [
        qw/API::REST RetryOnError OAuth/,
        AutoCursor => { max_calls => 40 },
        AutoCursor => {
            max_calls      => 5,
            force_cursor   => 1,
            array_accessor => 'users',
            methods => [qw/friends followers/],
        },
    ],
    # additional new arguments...
);

With the Net::Twitter object above, calls to friends will return a reference to an array with up to 500 users.

The AutoCursor role is currently in a developer release.

http://search.cpan.org/~mmims/Net-Twitter-3.16000_1/

UPDATED 2011-03-29: AutoCursor is now available in the production release with a minor change that allows the override of max_calls on a per-call basis:

$nt = Net::Twitter->new(traits => [qw/API::REST AutoCursor ...

$ids = $nt->followers_ids; # up to 16 calls; 80,000 ids
$ids = $nt->followers_ids({ max_calls => 40 }); # up to 200,000 ids

I would appreciate some feedback from those who will be affected by this Twitter API change before adding AutoCursor to the production release. Does it meet your needs? Is the interface sufficient? Please test it with your code and let me know what you think. You can comment here, shout at me on Twitter (either @semifor or @perl_api), or drop by the #net-twitter channel at irc.perl.org.

Twitter API calls with LONG timeouts

Is slow Twitter API throughput suddenly sucking the life out of your perl based Twitter application? Here’s a hint: set LWP::UserAgent’s timeout value to something sane, like 8 seconds. You can do that in the Net::Twitter constructor by providing a useragent_args parameter:

my $nt = Net::Twitter->new(
    traits => [qw/API::REST OAuth RetryOnError/],
    useragent_args => { timeout => 8 },
    # ...
);

Or, set it anytime after construction:

$nt->ua->timeout(8);

I recently noticed Twitter API calls taking an extraordinary long time to complete. Twitter was accepting connections for API calls, but many of those connections were simply timing out.

This is unusual. Twitter normally either returns a successful result, or an error in a reasonable amount of time. (My tests indicate 4-5 seconds.) But these unanswered requests were not getting timed out by Twitter—-they were getting timed out on the client side.

LWP::UserAgent has a default timeout of 180 seconds. That’s a very long time to wait. By setting the timeout to something more reasonable, like 8 seconds, I was able to limit the damage done to application throughput.

With a more aggressive timeout value, it makes sense to use Net::Twitter’s RetryOnError trait. By default, it retries up to 5 times when it encounters a temporary error. It classifies any HTTP response code greater or equal to 500 as a temporary error. It delays 250ms before the 1st retry, 500ms if that retry fails, and doubles the retry value on each consecutive temporary failure. So, there is a 4 second delay before the fifth and final retry.

I’ve found the vast majority of temporary failures succeed on retry with RetryOnError’s defaults.

RetryOnError is configurable. You can specify the initial_retry_delay, max_retry_delay, retry_delay_multiplier, and max_retries. You can also provide retry_delay_code, a CODE ref that takes a single floating point argument—-the number of seconds to delay. This is useful for logging retries, or for using something other than sleep to delay—-for example, Coro::AnyEvent::sleep in a non-blocking environment.

Given what I’ve learned with this recent connection timeout problem, I will likely provide a sane default value for LWP::UserAgent’s timeout in an upcoming release of Net::Twitter.

Last year at Chirp (the Twitter Developer Conference), one of the speakers said Twitter’s internal strategy is to fail quickly and re-queue requests. If they kill client requests that can’t be fulfilled by the backend in N seconds, I’d love to know the value of N. My testing indicates 4—5 seconds, but it would be nice to have a definitive answer from Twitter. Do you know what the internal policy is? If so, please let me know.

Scammers

This morning I received a message from a good friend:

It’s me, Dave. I really don’t mean to inconvenience you right now, I made a little trip to UK and I misplaced my luggage that contains my passport and credit cards, I know this may sound odd, but it all happened very fast. I need to get a new passport and a ticket, but I’m short of funds to pay for my ticket and other miscellaneous expense. Please, can you lend me some funds to get a ticket? I’ll be willing to pay back as soon as I get home.

Please respond as soon as you get this message, so I can forward you my details to send the funds to me, OR you can drop a message via the hotel’s desk phone if you can. The numbers are, +447045733705.

I await your response

Of course, it wasn’t from Dave, it was from scammers attempting squeeze some funds out of Dave’s friends.

I quickly replied (to Dave’s real address—-the scammers actually registered an address doubling the last letter of Dave’s actual email address):

These scammers should have picked someone whose friends (a) have funds to send, and (b) would send those funds instead of replying, “If you survive the swim home, I’ll buy you a beer!”

This was an interesting scam. A few days ago, Dave noticed the address book in his Yahoo account had gone missing. Now it seems apparent his account was hacked and the address book stolen. Without it, he can’t send a warning to his friends alerting them to the fraud. Hopefully, no one is naive enough to fall for the ploy.

If anyone loses more than the cost of a beer over this, perhaps they can claim it as an educational expense on their federal tax return.

jrn - edit today's journal entry

I used to enjoy keeping a personal journal, but I’ve fallen out of the habit. To make it easier, I wrote myself a small script, named it jrn and dropped it into ~/bin which is in my PATH:

#!/bin/sh
 # 
 # Edit a "journal file" for the current date with format YYYY-MM-DD.txt
 # in ~/writings/journal or the specified directory.

 EXT="txt"
 BASE=$(date +%Y-%m-%d)
 DIR=${1:-~/writings/journal}

 exec ${EDITOR:=vim} "$DIR/$BASE.$EXT"

Now, to start or edit an entry for the current day, I just run jrn. If I can to make an entry in a different directory for any reason, I can just provide the directory name as an argument (. will specify the current directory).

New Privacy and Favorites Features

New features

The Rat’s been busy implementing a few new features that have been requested, frequently. Let’s cover them, first.

  • Make all my bookmarks private

    This option does just what it says. Enable it, and all your bookmarks will posted to Delicious for your eyes only.

  • Make bookmarks with this tag private (private-if)

    Just want some of your bookmarks flagged private on Delicious? Enter a “private-if” tag and the bookmarks created for statuses containing that tag will be marked private.

  • Bookmark favorite statuses

    Bookmarking URLs in favorited statuses is a popular feature. Some users want to bookmark their favorite statuses, not just the links they contain. Enable this option the Rat will bookmark your favorites whether or not they contain links.

Apologies

The Rat seems to have made a programming error resulting in a failure to follow the timelines of 70 recent accounts. Finally, one of those users questioned the Rat’s integrity. After having his cheese ration slashed, the Rat found and fixed the problem.

If you’re one of those 70 users, the Rat is truly sorry for the error.

Email addresses

In the past, all that was required to register a Packrati.us account was a Twitter sign-in and a Delicious account for your bookmarks. When your account needs attention, the Rat needs a reliable way to reach you. An @reply works, sometimes, but it’s not ideal in all cases. Sometimes, 140 characters just isn’t enough. Sometimes more confidentiality is needed. And sometimes, users just don’t see those @replies.

So, Packrati.us now captures email addresses for new accounts. For existing accounts, adding an email address is optional but encouraged. And they are required before new features can be enabled. Consider it gentle coercion.

We won’t sell, share, or use your email address in any way except to communicate with you when your account needs attention or as required by law.

Announcement list

When you sign in to Packrati.us and share your email address with us, please check the box that subscribes you to our announcement list. You’ll get infrequent messages, like this one, describing new features, tips, and other information you’ll likely find interesting.

Just a fraction of the registered users have subscribed to the announcement list. That has the Rat confused. Packrati.us is mostly a register and forget it service. There’s no need to return to the website frequently. So, how are you going to stay informed about updates and changes to the service?

Yes, the Rat will post messages on Twitter, but the announcement list will be a more reliable way to get the news, especially if you sleep on a different schedule than the Rat.

Tip

  • Just want to bookmark URLs in your favorites, not your own statuses?

    Add an only-if tag. Set it to something crazy, like: #I_ONLY_WANT_FAVORITES_RAT. Then never use that hashtag in your tweets. If you have favorites bookmarking enabled, that’s all the Rat will save.

Happy bookmarking, packrats!

Net::Twitter and the death of Basic Authentication

On August 31, 2010, Twitter discontinued Basic Authentication.

That means, if you’re using a username and password with Net::Twitter or Net::Twitter::Lite, you will get “401 Not Authorized errors” attempting to access the Twitter API.

Twitter now requires OAuth authentication for access to the Twitter API. The good news is: Net::Twitter and Net::Twitter::Lite fully support OAuth. So the transition should be as pain free as possible.

For simple, automated scripts, all you need to do is register a twitter application, copy the consumer key and secret and the access_token and secret, and use them instead of username and password.

use Net::Twitter;

my $nt = Net::Twitter->new(
    traits              => [qw/API::REST OAuth/],
    consumer_key        => $YOUR_CONSUMER_KEY,
    consumer_secret     => $YOUR_CONSUMER_SECRET,
    access_token        => $YOUR_ACCESS_TOKEN,
    access_token_secret => $YOUR_ACCESS_TOKEN_SECRET,
);

$nt->update("Bob's your uncle!");

With Net::Twitter::Lite, it’s even slightly simpler:

use Net::Twitter::Lite;

my $nt = Net::Twitter::Lite->new(
    consumer_key        => $YOUR_CONSUMER_KEY,
    consumer_secret     => $YOUR_CONSUMER_SECRET,
    access_token        => $YOUR_ACCESS_TOKEN,
    access_token_secret => $YOUR_ACCESS_TOKEN_SECRET,
);

$nt->update("Bob's your uncle!");

Of course, with Net::Twitter::Lite, you don’t get all the bells and whistles, like the RetryOnError trait that vanquishes many Fail Whales. Or the InflateObjects trait that turns created_at dates into DateTime objects, and other goodies.

If you have a web application or interactive program that needs to obtain access tokens for multiple users, that’s easy, too. See the Net::Twitter examples directory or the Net::Twitter::Lite examples directoryin their respective distributions.

To stay informed, you should:

Find the latest version of this article on github.

Happy coding.