On using VPNs, Proxies and VoIP from behind a restrictive firewall

I framed my posts on setting up one's own secure, encrypted Internet access via a Raspberry Pi with the notion of accessing one's Netflix account from outside the U.S.  That's certainly an understandable desire, and either of the solutions I've documented so far (ssh proxy and pptp VPN) will work well for that purpose, assuming your hotel Internet connection provides sufficient basic throughput.

But there is another potential usage: travel through a region in the world known for suppressing or monitoring individual Internet activities.  For example, in one country I've visited whose national firewall is well-known, all blog engines I tried were blocked (blogger.com, wordpress.com), as were all user-generated video sites (YouTube, Vimeo) and all social-media sites (Facebook, Twitter).  Similarly many political and commentary sites were blocked, especially those with what Americans would regard as conservative or libertarian viewpoints.  Most news sites were allowed through but in localized form with filtered content.

In such cases, it may be helpful to have one's Raspberry Pi running 24/7 on your home broadband connection, serving up the several work-arounds I've documented:

1. An encrypted pptp VPN suitable for any client (Mac, Windows, iOS...);
2. A ssh proxy suitable for laptops, and
3. A php anonymous web proxy I haven't gotten around to blogging about yet.

(Just be very sure you're familiar with your host's rules and laws... see below.)

Here in the U.S., I've found each of these approaches to have pluses and minuses:

(1), the pptp VPN approach, has generally been preferable for my usage since all platforms are supported, all traffic can be encrypted, your IP address cloaked, and blockage of things like VoIP services is circumvented.  This approach requires no downloaded client or jailbreaking, but it utilizes the standard pptp IP port, which some localities block (and, annoyingly, it seems alternate ports cannot be specified in the built-in VPN client in Windows, OS X, iOS or even Linux).  Note, however, that pptp VPNs utilize the ms-chap v2 encryption algorithm, which some commenters here have pointed out is potentially hackable, though with considerable effort.  Using a long, highly randomized password can help on that point.

(2), the ssh proxy, is immune to the port-blockage problem because I've documented a way to use the port ordinarily used for https web pages, which presumably will probably never be blocked.  All your traffic is encrypted, your IP address is cloaked, and blockage of things like VoIP activity is circumvented.  No downloaded client software is needed.  But, it's suitable for laptops only.

(3) is a familiar anonymous web proxy, so it's just a web page and its port will never be blocked except in severe cases of national Internet shutdowns.  It requires no downloaded client software.  But only your web traffic is encrypted/cloaked.  Things like VoIP-port blockage can't be gotten-around using this approach.

Of course, there are also commercial services like Cloak (www.getcloak.com) and TorVPN (www.torvpn.com) and any number of anonymous web proxies available, both free and paid.  However, commercial services' IP addresses or URLs are sometimes blocked by institutions and countries.  Running your own service on your home broadband connection should avoid this issue if you keep your usage moderate and your mouth shut, and if you avoid the temptation of checking your cloaked IP address using services such as http://whatismyipaddress.com which can record such inquiries and thereby database them as likely coming from curious users of VPNs or proxies.

When it comes to countries with restrictive Internet policies and national firewalls, you will still probably find the pptp port accessible through your hotel connection.  Most host countries want to encourage commerce and more concerned about suppressing individual activities than preventing businessmen from securely contacting their corporate resources.  Consequently, the classical pptp VPN may be open, although known non-corporate VPN and proxy services may be blocked on a URL or IP address basis.

VoIP

Making calls when traveling in foreign countries can cost a fortune using conventional cell-phone roaming minutes.  You may find that even despite the crappy Internet connection in your hotel, the standard VoIP port may remain unblocked.  If so, and if it's legal where you are, you can try making a test call using your smartphone's VoIP app-- my favored iPhone/Android VoIP app, 3CXPhone, has worked beautifully for me in many such circumstances.  If so, the beleaguered router in your hotel has an effective Quality-of-Service (QoS) engine that prioritizes your VoIP traffic well enough to work for you.  Or, if the VoIP port is blocked (or if you'd prefer your conversations be encrypted, which normal VoIP is not, or if you'd like your use of VoIP to be hidden altogether), running your connection over your Raspberry Pi's pptp VPN might work, but the additional latency might or might not be a problem.  The new Silent Circle app might also be of interest if security and privacy of your conversations is desired; ditto running Liberte Linux or some other anonymized operating system.  Just keep in mind the legal points noted below.

A couple of important points

1) Technical

There are many potential bottlenecks in a cloaked or proxied connection.  Your hotel Internet connection is one, and nowadays it's almost a given that it will be pretty lousy.  But your home broadband connection is potentially another (especially the upstream bit-rate, which is usually much lower than downstream).  And the current Raspberry Pi implements its Ethernet port as a USB 2.0 device, so its throughput is limited to USB 2.0 speeds (theoretically 480 Mbit/sec but, in practice, less).  Keep in mind that every bit of VPN'd or proxied traffic must flow through your Raspberry Pi twice: first via your broadband downlink from the site our service you're visiting, and then via its uplink to you.  Plus, routing your Internet traffic through your Raspberry Pi from afar obviously adds latency.  All these limitations have an additive effect.

Also keep in mind that some localities will monitor traffic bit-patterns and flag unusual usage.  For example, if you're running a ssh proxy over the https port as I've documented, your host might notice that your usage doesn't match ordinary webpage-visiting behavior.

Your home ISP might similarly detect your usage (especially if it's heavy) and declare your online employment of the Raspberry Pi to be in violation of terms of service which prohibit running servers.

2) Legal

Accessing forbidden websites and services, utilizing a VPN or other forms of encryption, using a proxy, communicating via VoIP, etc., may be illegal in some countries, and many such activities can get you in trouble on college campuses too, not to mention the dim view employers take of employees accessing blocked services via whatever tricks. (It can even be cause for immediate termination.)

Please be mindful of your host's rules and laws-- this is your responsibility and obligation, and (not being a lawyer) I can offer no advice or counsel on the matter.  As far as countries go, while it's true that many are more concerned about keeping their citizens in line rather than crimping visitors' style, that's a risky assumption to make.  Be informed, and utilize tools such as these at your own risk and with your eyes wide open.

How to set up a real, encrypted VPN through your Raspberry Pi

Well isn't it always this way... the very day after I post on how to use the Raspberry Pi as a secure Internet proxy by working around a missing element in its variant of Linux, that missing element becomes available in the stock OS!

Here's how it went.  On a whim, even though I'd just given my Raspi a full software update just two days ago when I made that earlier post, I repeated the process again today:

sudo apt-get update
sudo apt-get upgrade

...and lo, there were some important packages that updated.  Sniffing further, I attempted the first step of the installation process for installing a "real" PPTP VPN onto Debian Linux:

sudo modprobe ppp-compress-18 && echo success

No error was reported!  The formerly-missing MPPE module has been added to the kernel.

Well now.  This poses the tantalizing possibility of setting up a real, Point To Point Protocol (PPTP) Virtual Private Network using the Raspberry Pi as a gateway.

A VPN strategy for every situation

Among the highly non-trivial benefits of the PPTP VPN versus the ssh proxy tunnel I documented previously include compatibility with iPhones, iPads and other mobile devices which might not offer ssh port forwarding capabilities in their operating systems.  That was the basic trick of the ssh proxy tunnel.

On the other hand, PPTP VPNs utilize the GRE 47 protocol which is some routers find indigestible while others can accommodate only one GRE 47 connection at a time and will block all others-- a recipe for trouble in hotels and public hotspots.  By comparison, the ssh proxy trick I documented--which leveraged the https or other common port for the ssh proxy tunnel--will be more likely to always work as you travel, at least for your laptops.  Frankly, at this point I have both the ssh proxy tunnel and the PPTP VPN set up on my Raspberry Pi, and I might as well leave it that way so I'm covered for just about any circumstances.

Here's how

There are many tutorials available for setting up the PPTP server daemon (pptpd) on a generic Debian Linux machine.  However, I've found that the Raspberry Pi presents a few quirks and needs its own instructions.  The following steps worked for me and resulted in a robust VPN that has held up to my usage so far-- this blog post was constructed entirely while connected via my Raspberry Pi's new PPTP VPN.

1. Open port 1723 on your router, pointing it to your Raspberry Pi's IP address on your LAN. 
2. Your Raspberry Pi should have a static IP address on your LAN.  Good instructions for that are here.
3. Next, if your home ISP does not give you a static IP address on the Internet, you'll need to set up an account with a dynamic DNS (DDNS) service.  This will give you a URL which will always point to your router.  My D-Link DIR-655 router (a truly excellent router, by the way) earns me a free DDNS account on D-Link's service; there are plenty of alternatives.  In D-Link's case, the url will look like [username].dlinkddns.com.  UPDATE: My trusty DIR-655 died, and the units currently being sold now are totally different.  I switched to an Apple Time Capsule, which necessitated a new dynamic DNS provider.  I chose DynDNS.org... you get one free host name for trying their 14-day "pro" account (which you can cancel, keeping the host name, though their services are powerful and quite cost-effective for what you get).  
4. Log into your Raspberry Pi either via its local console or via ssh.  
5. Issue sudo apt-get install pptpd
6. Issue sudo nano /etc/pptpd.conf --this will start a simple text editor on your Raspi.  Note that most lines are comments and are commented-out with a "#" at the beginning.  Scroll down until you find the lines which set localip and remoteip.
7. Set localip to your Raspberry Pi's IP address.  For me, that line becomes localip 192.168.0.202.
8. Set remoteip to encompass a small block of IP addresses on your LAN that will be made available to your remote client(s).  For me, that line became localip 192.168.0.240-255 ...there are various rules for how that block can be constructed; use the format shown here or it might not work.  
9. Hit ctrl-X and save the file.
10. Now issue nano /etc/resolv.conf and make a note of the nameserver identified in this file --for me, and I'd bet most other folks, it's the LAN address of my router, 192.168.0.1.  Hit ctrl-X to close the editor.
11. Issue sudo nano /etc/ppp/pptpd-options and scroll down until you find references that set ms-wins and ms-dns.  You want to set thise to the nameserver address you just noted.  For me, these lines became ms-wins 192.168.0.1 and ms-dns 192.168.0.1 respectively.  Hit ctrl-X and save the file.  This completes the basic IP setup for your new VPN.
12. Issue sudo nano /etc/ppp/chap-secrets and scroll down to an empty line.  Here is where you will specify your users who can use your VPN.  For a user named Ralph with a password of SuperSecret123, the line would simply be Ralph pptpd SuperSecret123 *  --the asterisk at the end is important as it specifies that Ralph can tunnel in from any IP address.  Repeat with additional lines to specify other users you want to allow to tunnel in via your Raspberry Pi.  Then hit ctrl-X and save the file.
13. Issue sudo nano /proc/sys/net/ipv4/ip_forward  ...see the zero?  Change it to a 1.  This tells your PPTP daemon to forward remote clients to the Internet.  Hit ctrl-X to save the file and exit the editor.
14. Issue sudo nano /etc/sysctl.conf and scroll down until you see the commented-out line that states net.ipv4.ip_forward=1  ...uncomment this line (remove the"#").  This will tell your Raspberry Pi to always allow remote clients to access the Internet via their tunneled connection.
15. You're done!  Reboot your Raspberry Pi.
16. While you're waiting for it to sort itself out, you can set up a VPN connection on your traveling (client) machine.  On the Mac, go to System Preferences | Network, click on the padlock and authenticate as an administrator user, Click the "+" sign and select "VPN" from the pull-down menu.  Give this new VPN a name.  Voila a new, unconfigured VPN connection will pop into your network-connections options.  With this selected, type in your dynamic DNS address for your Raspberry Pi (e.g., YourUserName.DynDNS.org).  In the Account Name field, put the username you specified in step 12 ("Ralph").  Click the Authentication Settings and specify the password from step 12 ("SuperSecret123").  Click the Advanced button, and be sure to checkmark the option to route all your Internet traffic through the VPN.

You're done!  (Setting it up on an iPhone or iPad is even easier.)  

You should be able to connect now, and all your Internet traffic will be encrypted and funneled through your Raspberry Pi from wherever you are... or at least from wherever VPN connections are tolerated.  For other places, consider the ssh proxy tunnel I wrote about before.

How to use a Raspberry Pi as a secure Web gateway from anywhere

UPDATE: The necessary modules for supporting a "real" PPTP VPN have been added to the stock kernel.  While the following methodology offers benefits in many situations, you might want to consider this new post on the topic, or set things up both ways.

----

Being a geek, I was the first kid on my block to sign up to purchase a Raspberry Pi Model B, a $35 computer-on-a-card that's about the size of a pack of cigarettes.  In fact, since it comes without a case, I'm tempted to buy a pack of cigarettes, toss the coffin-nails and use the wrapper as an enclosure.  Maybe someday.

Anyway, this is a fully-fledged computer centered on an ARM-based processor that's a couple notches down in capability versus the CPU found in an iPhone or Android phone.  Nevertheless it is capable of running a variety of operating systems, and it benchmarks roughly equivalent to a typical i386 or Pentium machine of a few years' vintage.  Against that metric, the power efficiencies alone are amazing.  This thing runs off a cell phone charger.

The Debian-based variant of Linux that the Raspberry Pi folks recommend as their standard OS now enables hardware acceleration which makes most operations nicely speedy.  It works quite well and, being Debian-based, has access to a huge array of free software.  I have a few projects in mind for this little pup and currently have it running the full LAMP webserver stack (Linux, Apache, MySQL, PHP) just for chuckles.  Despite all that, more than 180MB of its 256MB of RAM is unused.

There will be more to come for this device.  First up: sometimes in my overseas travels I find I can't access my US-based services like Netflix.  Connecting via a hotel WiFi network when overseas gives me a non-US IP address which Netflix detects, and since Netflix's licensing agreements with studios and TV networks don't cover overseas customers, it will refuse to serve up content even though I have an account in good standing back home in the U.S.  Blockage of other Internet-based functionalities also sometimes occurs for other reasons-- for example, VoIP traffic is sometimes blocked, either purposefully or through poorly-considered blanket policies which boil down to "We shall allow common things like web browsing and block all else."

How nice it would be, then, to run a VPN or similar private pipe through the Raspi.  The Raspi's IP address would then be the face of my connection to Netflix and everything else.  Theoretically, I could activate that whenever I needed a US connection or found my connection to some service blocked for whatever reason.

Unfortunately, the Raspi's current recommended Linux build contains a stripped-down kernel that does not support certain features needed for establishing a common encrypted PPTP VPN connection.  Specifically, the kernel lacks MPPE support, and the Raspi will spit an error in the first step of procedures like the one documented here.  Some folks have compiled custom kernels--it's not too difficult--but for various reasons I want to keep my kernel stock-standard.  Perhaps someday the stock kernel will include MPPE support-- it seems everything else needed is present in the stock build and its repositories.

But there's a simple alternative which works well and is almost as capable and easy to use, and that is to use the port-forwarding capability of the ssh server built into Linux to set up the Raspi as a proxy for remote machines.

Here's how

• First, the ssh server must be active on your Raspberry Pi.  Most folks will have enabled this at setup time.  If not, poster "Abishur" gives the details here.
• Set your Raspi with a static IP address on your LAN.
• Next, if your home ISP does not give you a static IP address on the Internet, you'll need to set up an account with a dynamic DNS (DDNS) service.  This will give you a URL which will always point to your router.  My D-Link DIR-655 router (a truly excellent router, by the way) earns me a free DDNS account on D-Link's service; there are plenty of alternatives.  In D-Link's case, the url will look like [username].dlinkddns.com.  
• You'll need to open your router's ssh port and forward it to your Raspi's IP address on your LAN.
• On your Mac, bring up the System Preferences | Network preference pane.  Click the padlock and authenticate as an administrator if necessary.  In the Location menu at the top of the preference pane, select Edit Locations.  Highlight the connection-location you normally use for attaching to networks when you travel, click the "gear" icon, and select Duplicate Location.  Give this duplicate a name such as "Normal Connection over ssh via Raspberry Pi".  Click Done.  Now, select the duplicate location and click the Advanced button, then the Proxies tab.  Checkmark the SOCKS protocol and specify "localhost" and port 8080 ...actually, the port doesn't much matter (just be sure to specify the same port in the next step), and 8080 is innocuous.   (Click to enlarge):

Click OK and then Apply.

You'll find your Mac will update its connection to whatever WiFi or Ethernet service it was on before, but Internet traffic won't flow quite yet.  Next, open a Terminal window.  If you've been working in a non-privileged standard user account, type "login [administrator account name]" and enter the password at the prompt.  Then--and here you need to be a bit cautious since you're futzing with your Mac's system files--type "sudo nano /etc/ssh_config"

This will start the Nano text editor and load in your Mac's ssh defaults.  Scroll down to just above the "Host *" line and insert the following:

Host webproxy
  HostName [your dynamic dns url]
  User [your username on your Raspberry Pi]
  DynamicForward 8080
  Compression yes

Type ctrl-X and say yes to the prompt to overwrite the current copy of ssh_config.

Close the terminal window.  Setup is now complete.

Using your secure tunnel connection

Now you're set.  On any given day when you find an Internet service blocked:

1. In the System Preferences | Network preference pane, select the location you created with the ssh connection proxy via your Raspberry Pi.
2. Open a terminal window, and simply type "ssh webproxy" and provide your Raspberry Pi user account password when prompted.  

You'll now have Internet!

...But don't close that terminal window.  What's happening now is that all TCP traffic to and from your computer is being routed through your encrypted ssh connection, which will be closed if you axe that window.

When you're done: switch back to normal connectivity by selecting your usual location in the System Preferences | Network preference pane.

Note that this approach is almost but not quite as functional as a nice PPTP VPN.  It is, however, pretty much limited to providing remote access just for personal computers.  Connecting via ssh is not well-supported by mobile devices, unfortunately.  For those, a proper VPN setup would be most beneficial.  Perhaps next time.  Also, it seems that while TCP traffic is routed, some other types of traffic may not be (which would seem to be why you can type "ssh webproxy" and log onto your Raspberry Pi when Internet isn't otherwise flowing yet after selecting the ssh proxy'd location in the preference pane.)  So even though TCP traffic is encrypted, don't go trading state secrets over this connection.

In my testing, using this ssh proxy approach has shown a negligible impact on Internet throughput versus my home's normal network connection, which is admittedly fairly entry-level.  If my hotel's Internet connection is good enough to support Netflix streaming in the first place, content streams just fine.  Latency increases a little, but it's pretty terrible to begin with on my home Internet service.  Note that all your remote traffic routes through the ssh port on your router, so if your remote usage would involve time-critical applications like VoIP, it might be a good idea to add that port to the router's Quality of Service (QoS) priority list.

Demo: We're Not In Kansas Anymore, Toto

With my Mac tethered to my iPhone, the handy WhatIsMyIPAddress.com maps my IP address as in Kansas:

By selecting my new ssh proxy "location" in the System Preferences | Network preference pane and typing "ssh webproxy" in a terminal window, now it appears I'm connected via my home ISP in California, where my Raspberry Pi resides... even though I'm still tethered to the iPhone:

One wrinkle...

You should consider the possibility that in your travels, access to the ssh default port of 22 might be blocked as well.  A good plan for that case is to make use of a port unlikely to be blocked and otherwise unused in your setup.  For me, although I'm playing with my Raspi as a tiny web server, it's not serving encrypted (https) pages.  So the https port 443 is not being used and is a great candidate for our pipeline's usage.  To use this:

• Open port 443 on your router and forward it to your Raspberry Pi.
• If you have Apache enabled on your Raspberry Pi, comment-out the "SSLEngine on" line in /etc/Apache2/sites-available/default-ssl:

•  Then add port 443 to the listen-list in /etc/ssh/sshd_config on your Raspberry Pi:

Reboot your Raspi ("sudo reboot") and give it a couple minutes to sort itself out.  Henceforth, on your Mac, you can type "ssh -p 443 webproxy" to route your ssh connection over port 443, which all but the most uncooperative hotspots should leave unimpeded.


Incidentally, most of the above can be useful for figuring out how to do something similar on alternate platforms: a Windows remote client instead of a Mac, for example, or a generic Linux machine or something like a Mac Mini rather than a Raspberry Pi.  Of course, if you can use a fancier machine than a Raspberry Pi (or use an alternate or custom kernel in a Raspberry Pi), you can just set up a nice PPTP VPN on it.  If the ssh proxy approach is any indication, it should work just fine.

How to do cross-references in Scrivener

One habit I've brought to Scrivener from Word is to insert graphics and tables as I write.  I like to include cross-references to them in my text as I peck along.  Word has an effective way of doing that via the Insert | Caption and Insert | Cross-Reference menus.  An advantage is that numbers are automatically updated and regenerated (at least at print-time) if you move things around.  Disadvantages to Word's implementation include limited call-out flexibility (for example, you can label a figure with "Figure" but not "Fig." or "FIGURE" as some publications mandate).  And there's Word's annoying habit of forcing you to wade through much of the full menu with each entry, and of course there's always the Random Format Generator that seems to kick in whenever you ask Word to do anything.


After much googling it seems there is a way of doing something quite adequate in Scrivener to support cross-references.


What the Scrivener folks have implemented in recent releases is an expansion of their Auto-Number functionality, which let you insert a placeholder (such as <$n> to specify an Indo-Arabic numeral placeholder) that, on compilation, would be filled-in with a number in proper sequence.

The placeholder code can now be appended with a type code for the cross-reference, and a label of your choice.  For example, expanding on the Scrivener draft I started in my last post, I've pasted in a couple of graphics from one of my source references (a time-series graphic and some statistics stuff) and typed in cross-reference codes of my own choosing to help me keep track as I write.

Here's how it looks as I compose my draft in Scrivener (click to enlarge):

Note the cross-reference callouts, <$n:figure:time_series> and <$n:figure:statistics_stuff>.  These will render only as numbers when compiled.  The numbers will be in simple order of appearance in the document.  Although codes such as these are a departure from WYSIWYG and may seem a little WordStar-ish, they do have the benefit that I can quickly see that they reference figures, and that one references the time series while the other references the statistics stuff.  That's actually nice for composing-- in Word's way of doing things, I have to scroll down to see what "Figure 1" points to.

Here's how it looks when exported ("compiled") to Word:

And here's what happens when the citations in the text are reversed... note the citations are renumbered in order of appearance.

How to use Zotero to make citations easy in Scrivener

I'm spinning up a lot of writing lately, and I'm finding Scrivener to be a marvelous tool for getting drafts accomplished.  It's founded on a nifty notion: by separating the composition and presentation of written work, both are facilitated.

Not only is Scrivener more stable and less intrusive for writing than Word, but it has built-in organizing and bookshelf capabilities that help me collect necessary background information, get my thoughts sorted, and chapterize and reorganize things as makes sense.  It's easy on the eyes and very thoughtfully put-together-- obviously a tool constructed by a writer for writers.  And, once my draft is done, it exports to Word and other formats beautifully, so editors or colleagues don't need to use Scrivener also.  (However, it's available for Windows now, so those editors-and-colleagues are missing a treat if they don't.  Even Linux users can get in on the fun.)

One thing Scrivener (and Word) is missing is slick citation and bibliography management.  Scrivener natively supports EndNote, a Thomson Reuters bibliography-management package.  But that solution costs several times what Scrivener itself does.  I'm sure it's a very powerful solution, but for me it's overkill.

By comparison, Zotero is a free, cross-platform, open-source, cloud-enhanced "personal research assistant" that performs critical citation management and bibliography functions.  I've used it off and on for years, but until recently it supported only the Firefox browser.  I much prefer Safari (yay, Reader mode!  yay, ctrl-shift-I to email an article!), so it was with delight that I discovered Zotero is now available as a stand-alone app and supports Safari via an extension.  http://www.zotero.org/support/3.0 has all the download options; scroll down for the Zotero Standalone app and the Connector extensions for Safari and Chrome.

So here's a straightforward workflow to start with.  (Click the photos for a closer view.)

1) Gather your research.

Start with Zotero open.  Select (highlight) the folder into which you want your citations collected.  In Safari, browse to source articles.

Most scholarly publications (and many others, including web pages) are automatically supported by Zotero.  You'll see the Zotero button to the left of the omnibar turn from  to  to indicate the article is capture-able by Zotero.  Click it, and the citation will automatically be added to your folder in Zotero.

2)  Add your source items to your Scrivener project for offline access.

Now drag the little icon next to the URL in Safari's omnibar into your Scrivener project.

...Now you have the citation information in Zotero and the source item in your Scrivener project binder.  (If it's a web page, it'll be stored fully rendered-- absolutely marvelous for making productive use of airborne hours.)  As you write, ctrl-opt-F lets you search your entire project for keywords-- a boon.

3)  Write.

What I've been doing is spewing words onto the draft, and when comes time to insert a citation I just type two brackets [] and then drag the citation into them from Zotero:

Then, select the citation and make it into an inline footnote:

 The citation remains readable to me as I write:

...but it will turn into a footnote when I export (compile) it to send to an editor or colleague:

Here's what it looks like in Word:

  

You can get much fancier.  If, for example, you need to submit your manuscript to several places which might mandate different citation formats, blogger Liz at Confectious.net has documented a very slick way of putting textual cites in your manuscript in curly braces and then having Zotero scan, format and output your work with the necessary citation format.

New tool to update your iPhone 4/4S to iPhone 5!

If the rumors are true...

Then I plan to make a mint selling these:

One Kickstarter project coming up!

Check and protect your Mac or Linux computer from the "Flashback" Java trojan

Prior to OS X Lion, a Java interpreter came installed with OS X.  This allowed programs written in that language to be run; most often this enabled some web-related functionality.  Unfortunately Java seems to have recently stolen Adobe Flash's crown for vulnerability.  Successive security whoopses have driven Apple to remove both Flash and Java from its as-delivered builds of OS X (and of course neither was ever allowed on iOS).

That's great for folks buying new Macs and declining to install Flash and Java when requested by a website or software package.  But many (even most?) users do have these installed.  And a big Java vulnerability was discovered a couple of months ago.  Dubbed the Flashback trojan for its original packaging as a fake update to Adobe Flash (and how ironic is that?), it started life by infecting users' machines when they agreed to the seemingly-benign update.

Although Oracle, Java's owner, delivered a patch with reasonable speed, it took Apple seven weeks--and a disturbing outbreak of machines tainted from having visited infested websites, resulting in OS X's very first botnet--to issue an update.  The episode is a black eye for them.  It's an inauspicious performance by the internationally-regarded security guru they hired early 2011 as Global Director of Security.  And it's a sorry performance by the security-products industry as well, as the trojan flew under its radar the whole time, and then security firm Kaspersky Labs' removal tool turned out to have some unfortunate side-effects and was quickly withdrawn.  What a mess!

An official Apple remedy is available now, however, and given the severity of this outbreak it's essential that you update your Mac to eliminate the possibility of infection.  Do this even if you ran the terminal commands published by F-Secure, as there are reports of folks who'd been given the all-clear but later learned there was indeed an infection on their machines.

1. First, check to see if your machine even has Java installed.  Open Safari, select Preferences from its menu, click the Security button, and see if you have an "Enable Java" check-mark available.  If not, you don't, then you don't have Java installed, and you win a beer.  If so, seriously consider un-checking it, as Java is rarely required today in this day of HTML5.  Note that despite its similar name, JavaScript is something else and is unrelated to this vulnerability.
2. Next, close your browser and run Software Update.  Your Mac will churn for a moment or five, then ask for your administrator password, and the update (and any others) will install.  If the trojan is installed, it will be removed, and your Java installation will be updated with a less-vulnerable version.  (Still... uncheck that check-mark.  Seriously.)
3. If, in Step 1, you found you did not have Java installed, out of an abundance of caution Apple has still made a Trojan-checking/removal tool available.  Go to http://support.apple.com/kb/HT5246 and download and run the tool they've made available.  You only need to do this if you do not have Java installed.

Non-Mac victims a concern

Ominously, Ars Technica reports that while the Mac was the most prominent victim of this malware, a couple percent of the victims logged by Kaspersky Labs were Linux, FreeBSD and Windows machines.  This isn't surprising, since Java's original allure was for write-once/run-everywhere universality, and in fact there's nothing platform-specific about Oracle's patch description.  

The small number of Windows infections is very probably a testament to the automatic update mechanism Oracle and Microsoft instituted for Windows.  An attaboy to them, then.  

More worrisome is the approximately one percent of victims logged from the free/open-source OSes.  It's not at all clear what to do or where to turn for checking and disinfecting those machines.  Towards the bottom of the Oracle patch-page previously referenced is a sizable risk matrix; as poster "UnSpawn" on the LinuxQuestions.org site helpfully notes:

The Oracle page also contains a list of CVE identifiers. So if you have a CVELIST=$('links -dump $URI | awk '/\| CVE-20/ {print $2}'|xargs;') then depending on your distribution you could check if those require fixing and if they are yourself. Per-CVE details are at http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-yyyy-nnnn (or www.cvedetails.com/cve/CVE-yyyy-nnnn/) for Red Hat / Centos / Scientific Linux see https://access.redhat.com/security/cve/CVE-yyyy-nnnn (or 'yum --cve CVE-yyyy-nnnn'), for SuSE see support.novell.com/security/cve/CVE-yyyy-nnnn.html, for Ubuntu see people.canonical.com/~ubuntu-security/cve/CVE-yyyy-nnnn, for Debian and .*BSD see http://cvechecker.sourceforge.net and for others, well, you either know how to find your distributions SO bulletins or CVE listings yourself already or your distro maintainer(s) simply may not care.

At a minimum, update Java if you have it installed and constrain its activities as much as possible by taking a close look at your browser preferences, up to and including disallowing automatic execution of any scripts at all.  I'd imagine that antivirus utilities like Clam will now be watching for Flashback on the open-source platforms, too.

It's all a lesson to mind keep your machine updated whatever its OS, watch where you browse, and think carefully before installing software--even an update--unless you're certain of its source.

Another thing to consider is disabling system-level Flash and Java from your machine but keeping a copy of Google's Chrome browser handy.  Then you can rely on the built-in Flash and Java plug-in for that browser on those occasions when you need them.  This is what I do now-- I rely on Safari for 90% of my web work out of preference for its support of OS X Lion's gestures and its invaluable Reader Mode (with its nifty one-click send-article button, which emails articles all nicely formatted with source links and subject lines all filled-in).  Come across a site that requires Flash or Java?  Just copy the URL over to Chrome.  Best of all worlds.


UPDATE: Another Java-based trojan has been detected.  Called Backdoor.OSX.SabPub.a or SX/Sabpab-A, it is stymied by the latest Java patches (and of course by removal or disabling of Java).  One may surmise that Java vulnerabilities are all the rage among malware writers right now.  Another reason to take this seriously, then.

How to back up your Gmail and other Google stuff for safety, economy or switching

Whether you're a happy user of Gmail, Google Docs or other Google goodies, or a privacy-concerned current user considering alternatives but shackled by your stash of stuff on Google's servers, having local copies of your Google-stored possessions is valuable, wise and freeing. Loyal users can take comfort in having their data available offline on their own machines and backed up by Time Machine or whatever backup process they employ. Switchers will find their options broadened if their data is in their own hands and less subject to profiling or tracking. And those continuing with Google services can keep their service free if they periodically download their Google items onto their own computer and purge them from Google rather than paying an upcharge for exceeding the free storage threshold.

But Google doesn't make it easy to extricate your data from their grasp. After all, the more of your stuff resides on their servers, the stickier their services are for you, and the more data they have about you for targeting ads to you, and more.

Gmail poses particular difficulties for downloading your data in useful form, and it's the Google service most likely to bloom out of control, as it has for me.  But there are several ways to get your data into your own possession.

Here's how:

A) Use an email client

For downloading useful copies of all your Gmails you can in principle set up Mail.app, Thunderbird or some other email client to sluice it all down. But that takes a lot of time, and if your emails go back years you can choke those applications. I found Thunderbird prone to cataclysmic thrombosis when I tried using it to back up my out-of-control Gmail account, and even OS X Lion's fantastically-improved Mail.app labored mightily under the burden of keeping local copies of my multi-year Gmail inbox and all my label folders.  Plus, somehow my 16GB of Gmails turned into 118GB on my disk in Mail.app, due to (1) inefficient, repetitive downloading of emails in label folders, and (2) the Spotlight-searchable and Time-Machine-friendly single-email storage approach that Apple instituted several years ago.  By comparison, Thunderbird, Outlook and many other email clients use a database approach to storing emails.  This has the drawback of emails not being Spotlight-searchable, and Time Machine has to back up the whole database whenever a single email arrives, but it's space efficient.  Mail.app's approach is searchable by Spotlight and backup-friendly, but since each email is its own file, each consumes at least the file system's minimum file size, which I believe is 4kbytes.  The problem is exacerbated when your Gmail account has many labels-- and let's face it, labels are a key reason for using Gmail.  But if you're not careful (and what documentation exists on this point is pretty unclear) you can end up having multiple copies of emails bulging your disk.  Bottom line: using an email client is a Gmail-backup solution for simple and small Gmail setups only, and even then it requires lots of time, and careful setup not to make a mess of things.

B) Use a service

Backupify is another possible solution for getting hold of your Gmails (https://www.backupify.com/). It's a service-- it does the pull from Google's servers on its own machines, then sends you a .zip. The cost is low, $3 a month, and you can do a one-time snapshot for free. It looks like a great option, especially for enterprises reliant on Google's services.

C) Use a tool

CloudPull's setup is super-easy.

An even better solution for individual users and many businesses is CloudPull, an app that runs on your Mac and backs up your Google emails and documents in a format that's nicely databased but still plays well with Mail.app, your other applications and Time Machine. It's published by Golden Hill Software (http://www.goldenhillsoftware.com/) and is available for $24.99 directly from the Golden Hill site or via the Mac App Store. It'll do incremental backups of all your Google stuff, automatically, up to hourly. It maintains point-in-time snapshots for a default of 90 days. It can handle multiple Google accounts. You can select which services to back up-- Gmail, Docs, contacts, calendar, even Reader.

And since it's an app that runs on your own machine, you don't have to proffer up your passwords to a third party or run your data through anyone else's machines.  By default, it loads every time you boot your computer and will incrementally update your local store automatically.  It works great if your disk is protected by FileVault 2, too.

CloudPull really puts you back in charge of your data. Your data will no longer serve as a hostage that limits your mobility and choices. 

CloudPull lets you back up
the full gamut of Google services.

I've been using CloudPull for a few weeks to back up my Gmail account while I ponder the possibility of either switching, clearing out my stuff from Google's servers to avoid another year's storage fee, or otherwise adjusting my usage of Google's services.  CloudPull has a nice, clean, modern Mac user interface-- it was written using Apple's up-to-date Cocoa framework, is 64-bit, and requires OS X Lion. There are nice interactive pointers for setting Google options correctly for its usage. It's nice, period. Its good first impression has maintained throughout my use of it.

CloudPull stores your emails' metadata in a self-contained, space-efficient SQLite database for quick searching.  It pulls down each email's text and attachments separately in individual files for backup-friendliness.  Your label/folder format is preserved.  CloudPull provides its own fast search facility.  Or, you can export any label (folder) to a standard, Mail.app-compatible mailbox format, compatible with Spotlight.  

CloudPull can store your snapshots
for a selectable about of time.

On first run, CloudPull will ask for your Google account ID and password.  Next, you check-mark which services to download to your computer.  For Gmail, you're presented with a list of all your labels, which you can checkmark for downloading.  The download then begins, starting with CloudPull acquiring a comprehensive list of emails in your account.  CloudPull always uses a secure, SSL-encrypted connection for communications with Google.

Depending on how massive your Gmail store is, it may take time for it all to trickle down the first time.  In my case, over 300,000 emails comprising more than 16GB needed to be downloaded-- quite the severe test of CloudPull's stability!  This took three days.  It ran without a hitch, though.  I do recommend leaving it alone and resisting the temptation to play with its search facility or scrolling through its window until it's finished.  The current version (2.02) doesn't provide a lot of progress feedback, so just let it run until the little cloud icon in the menu bar indicates downloading is complete.

Interruptions are handled gracefully-- if you should restart your computer, sleep it or lose your WiFi connection, it will resume by checking with Google's servers and recommencing where it left off.

Once all is downloaded, you can tell CloudPull how often to back up your Google data.  Incremental backups take very little time.

Export any folder into a Mail.app-compatible,
Spotlight- and Time-Machine-friendly archive.

You have the option of keeping your Gmail emails in your CloudPull database or exporting them ("Restore as mailbox") to disk.  If your aim is to reduce your Gmail storage footprint or switch, you may want to export your inbox or important label folders since anything deleted from Gmail will remain in your CloudPull database for only the time you have specified for retention.  The default is 90 days, and though you can set this to "indefinite" it's probably safest to export.  

You can view your emails from within
CloudPull using its search facility
and snapshot browser.

If you haven't discovered Cover Flow for
Spotlight-searching old emails, you're in for a treat.

Though CloudPull reliably and unobtrusively copied my ridiculously overstuffed Gmail account onto my hard disk, according to the developer an update is planned/upcoming/imminent which will make handling really large archives like mine even better.  CloudPull support has been very responsive, too.

Recommended.

P.S. -- An absolutely wonderful usage totally aside from backup and switching

A few years back I was deposed in a lawsuit.  All my emails regarding the matter under litigation had been requested.  It took me a long, long time to find and collect all those emails in Outlook, which I'd been using at the time.

CloudPull combined with Gmail would have been a boon: You can create a filter in Gmail, make a label for it, and then CloudPull will automatically pull all those emails down to your machine.  Export those to a mailbox, and each will be in its own neat file.  Voila: a task that took hours and hours to satisfy a court order could be accomplished in less than ten minutes.

Is Roll-Your-Own-VoIP the Best Investment Available Today?

I finally ran the numbers.

I now know what the true damages are for having cut my landline phone service and rolled-my-own Voice-over-Internet (VoIP).

My most boring blog graphic ever.
Until you look at the final number.
Yowza!

First there is the investment.  About fifty bucks for my recommended analog-telephone adaptor, the compact OBi100 or OBi110 units, available from Amazon.  These can provide telephone service to your whole house (just be sure to disconnect your landline from the phone company's drop outside your house first).  It's easy-- you can figure it out on your own, or you can follow my instructions.

Then there's the option of porting your existing phone number from your landline phone company to a VoIP wholesaler such as voip.ms, with its exceptional service, great support and dirt-cheap per-minute rates.  That's $25.  (You don't need to spend that if you're setting up a new number.)

All told, on Day Zero that adds up to a whopping $75 or so as up-front investment.  And you'll have invested maybe fifteen minutes of your time to plug it all in.  (Oh, there's the $25 you must put on-deposit when you open your VoIP service account, but that's refundable if you don't use it.)

The savings are immediate.  For us here in California, we're saving typically $35 a month versus our old landline.  And that's not counting benefits like the ability to make and receive calls cheaply via WiFi from anywhere in the world using our smartphones, or how our home is now firewalled against annoying robocallers and telemarketers, or how voip.ms optionally sends us our voicemails by email and lets us set up dedicated extensions (subaccounts) for free.  The voice quality is exceptional, and everything works just as it always did with our landline service.

And so I strapped on my MBA hat and finally ran those numbers.  I set a five year horizon, though it's almost been two years for us already, with that $35 profit rolling on without any drama, so I can't imagine a good reason to ever switch away.

I assumed a 31-day month to keep the spreadsheeting easy.  Assuming it all ends in five years, I'm seeing a 9000% annual percentage yield.  Twice, since we did this at two sites.

Not too flippin' bad.   9000% guaranteed yield in this day and age, in this economy.

After-tax!

If you know of a better investment, I urge you to email me privately straightaway, and please don't tell anybody else.

The easiest Roll-Your-Own VoIP yet

Since starting this blog, I've documented many reasons why roll-your-own internet telephony is a hugely cost-saving and flexible approach for home and business use for those who don't mind a small amount of initial geek-work.  Savings versus conventional phone service and digital providers like Vonage approach 90-95%, yet the quality is at least as good and the flexibility is far superior.  So I set up my two locations' phone services using the Linksys/Cisco PAP2T-NA analog telephone adaptor, which lets me provide high-quality, dirt-cheap phone service throughout my house (after disconnecting the PacBell line) just by plugging the adaptor into a phone socket and signing up with a voice-over-internet (VoIP) wholesaler like voip.ms or Callcentric.

The PAP2T-NA has served us well, but as a product it seems to be suffering from a measure of neglect after a sequence of corporate transfers: as I understand it, the unit was first designed and built by a talented startup named Sipura, which was purchased by Linksys, which was purchased by Cisco.  In any acquisition circumstance it's common that the acquired products become the red-headed stepchildren of the acquiring company, so it's understandable that there's a sense of the PAP2T-NA looking forsaken and wide-eyed as it begs for a scrap of sustenance from its current corporate parent.  Firmware updates grow few and far between, software utilities become incompatible with advancing computers and operating systems, documentation grows suckier over time, support web pages go un-updated and are difficult to find, links expire, and so on.  All these symptoms are regrettably the case with the PAP2T-NA now.  The firmware update process, for example, requires a Windows machine (assuming you can find fresh firmware in the Linksys/Cisco labyrinth).  Mac users are out of luck unless they spin up a virtual machine as I did.  Meanwhile the setup user interface is clunky and beset by the acronym-itis that afflicts VoIP.  Fortunately, there are tutorials (such as my own) which walk you through the setup process.  But it should be easier.

So it was with considerable interest that I read about the founders of Sipura starting up a fresh new venture to build easy-to-use VoIP hardware for Everyman.  Their new company, Obihai Technology, manufactures what can be regarded as the technological and spiritual successor to the PAP2T-NA, and it offers lots of additional functionalities and service possibilities, including such service possibilities as Google Voice and their own associated calling service, OBiTalk.  They even offer apps for iOS and Android devices.

Lots of possibilities there.  Plus, the company is fully supportive of "Bring Your Own Device" (aka roll-your-own) VoIP.  Which makes their new adaptor ideal for a review here.

The OBi110 is available at Amazon

The company kindly provided their compact OBi110 adaptor for me to play with.  It's about the same size as the PAP2T-NA, meaning about the size of two packs of playing cards.  Connection could not be simpler: there's a cable to attach to your router, and a port for your phone (connected, in my case, to my home's phone wiring.)  This model has an additional port for connection to a conventional phone line if you still have phone-company service; in that case the unit will switch between both services intelligently.  The whole purpose of our VoIP adventure is to eliminate that and its monthly costs, so that port is not connected to anything in our case.

As with any device of this breed, there's an impressive array of parameters-- some might say bewildering.  Fortunately, very few of these need fiddling when setting up with a VoIP wholesaler.  I'm a happy voip.ms customer, and that company provides a simple and helpful guide to setting the very few parameters that need changing.

The OBi110's web interface is clean and sparse, and it's easy to navigate since it uses a familiar expandable-tree motif for organization.

For use with my voip.ms account (whose setup I documented in my very first post on roll-your-own VoIP, here) only my voip.ms (sub)account number and my selected voip.ms server address needed to be entered.  A small quirk: you must uncheck the "Default" box to the right of any box you fill in, and you must scroll down to the bottom of any page you modify and click the "Submit" button before navigating to a new page.  In the example shown, I unchecked the Default box and typed in the identifier for the specific subaccount (i.e., phone-line) that I'd configured on the voip.ms site for use with my home line.  In my case, this identifier is "[MyAccountNumber]_home".  Scroll down, click submit.  On clicking the submit button, a notice appears that the change has been stored but the device must be rebooted for it to take effect.  Okay, point noted: there's a "Return" button to send you back so you can make more settings.  One simply iterates to enter in the required settings on each page, then submits that page, then return, and so on until all the needed settings have been entered.  Finally, there's a reboot button.  Click that, and within a few seconds your new settings will have engaged.

For voip.ms, the sum total of your settings is:

• You enter their server address in two places, 
• Your (sub)account number in two places, and 
• Your voip.ms password in one place.  
• If you're smart, you'll change the OBi110's web password.  
• Finally, voip.ms recommends going to the Physical Interfaces page and setting the Phone Port to default to SP1, meaning the service provider (in my case, voip.ms) set up in the first step.  There are fancier options as well.

Throughout, the OBi110's setup pages offer little question-mark symbols which, when you hover over with a mouse, provide definitions and hints.  That is a huge step forward for newbies and those wanting to stay afloat in the sea of VoIP acronyms.

Here's a specific example illustrating how much friendlier the OBi110 is versus the older PAP2T-NA design.  Like most people, I'm sure you've been annoyed at the fiddling the U.S. Congress and other governments have been doing with Daylight Savings Time.  In the past few years there have been several changes which our digital gizmos need to accommodate.  In the case of the PAP2T-NA, you can accommodate Congress' creativity by typing-in arcane character strings to define the begin and end time of your region's time-change:

Good luck figuring that out-- and in particular, good luck figuring out what needs to change next time Congress gets frisky.  The OBi110, by comparison, gives you simple date fields:

...Easy!  Except, that format might look puzzling.  No problem, just hover over the question-mark symbol, and up pops the information you need:

Pop-up help and definitions represent  a significant step forward in usability

...Of course, in the case of the fresher OBi110 design, the defaults were correct and needed no change.  I've seen PAP2T-NAs delivered with a variety of settings, both current and obsolete, and figuring-out and correcting that string of gobbledegook was left as an exercise for the user.

All told: armed with voip.ms' instructions, it took me less than five minutes to get the OBi110 set up.

We've been using this system for a while now and have been entirely satisfied with the sound quality and ease-of-use of the OBi110.  Firmware updates couldn't be easier: if you're taking advantage of the OBiTalk service (which I haven't yet explored), the process can proceed automatically with a couple of clicks.  Or, go to Obihai's support page and download the firmware from there.  Follow the instructions-- any computer and any browser running on any operating system can perform the update.  That's as it should be.

Summarizing: the OBi100/110 adaptors are my new top recommendations for homes and small businesses needing a single phone line.  The OBi110 has unusually flexible capabilities for bridging a variety of telephony technologies ranging from VoIP wholesalers like voip.ms and Callcentric to Google Voice and even your legacy phone service, so your single phone line can partake of the best and cheapest technologies.  These adaptors are easy and quick to set up and are agnostic towards your computer and OS.  Once set up, no computer is needed for everyday use.  These units are inexpensive to purchase and well-supported, and designed by people whose past work remains well-regarded and has stood the test of time.

UPDATE: A correspondent asks if the Obihai adaptor is compatible with the live VoIP-status monitoring trick I documented in another post.  Answer: Sure-- that trick queries the voip.ms server and does not interact in any way with the adaptor.

UPDATE: As is typical of analog telephone adaptors, the OBi110 provides a wealth of flexibility for configuring how outgoing calls are dialed and routed, for example among its various ports.  Importantly, its default configuration routes 911 calls to the plain-old-telephone-service (POTS) line, which it assumes is connected.  But since the whole point of Roll-Your-Own-VoIP is to get rid of POTS services and all their costs, the OBi110's default configuration needs to be changed to route 911 calls properly.  (This also assumes you have subscribed for your VoIP provider--e.g., voip.ms--to provide "e911" capability, which automatically sends your address to your local emergency services provider.  Now, the practice in the industry is to specify the call routing via strings of characters, which gives a lot of flexibility at the cost of incomprehensibility.  In the case of the Sipura/Linsys/Cisco PAP2T, these strings are called "call plans"; in the case of the OBi110 they're called "DigitMaps" and there are several of them.  For most situations the one you'll want to modify is the OutboundCallRoute one.  Open up the OBi110's configuration webpage and navigate through Physical Interfaces -> PHONE Port -> OutboundCallRoute.  In this field you'll see a bunch of rules, each contained in {curly braces}.  Look for the one which states, "(<#:>|911):li"  ...What this tells the OBi110 is: "If the number dialed is '911', dial it out the POTS (line) port."  Change "li" to "sp1" and it will dial your 911 call over your VoIP service provider instead.