| 104 |
Sitewide Search On A Shoe String |
One of the questions I got a lot when I was building web sites for smaller businesses was if I could create a search engine for their site. Visitors should be able to search only this site and find things without the maintainer having to put “related articles” or “featured content” links on every page by hand.
Back when this was all fields this wasn’t easy as you either had to write your own scraping tool, use ht://dig or a paid service from providers like Yahoo, Altavista or later on Google. In the former case you had to swallow the bitter pill of computing and indexing all your content and storing it in a database for quick access and in the latter it hurt your wallet.
Times have moved on and nowadays you can have the same functionality for free using Yahoo’s “Build your own search service” – BOSS. The cool thing about BOSS is that it allows for a massive amount of hits a day and you can mash up the returned data in any format you want. Another good feature of it is that it comes with JSON-P as an output format which makes it possible to use it without any server-side component!
Starting with a working HTML form
In order to add a search to your site, you start with a simple HTML form which you can use without JavaScript. Most search engines will allow you to filter results by domain. In this case we will search “bbc.co.uk”. If you use Yahoo as your standard search, this could be:
<form id="customsearch" action="http://search.yahoo.com/search">
<div>
<label for="p">Search this site:</label>
<input type="text" name="p" id="term">
<input type="hidden" name="vs" id="site" value="bbc.co.uk">
<input type="submit" value="go">
</div>
</form>
The Google equivalent is:
<form id="customsearch" action="http://www.google.co.uk/search">
<div>
<label for="p">Search this site:</label>
<input type="text" name="as_q" id="term">
<input type="hidden" name="as_sitesearch" id="site" value="bbc.co.uk">
<input type="submit" value="go">
</div>
</form>
In any case make sure to use the ID term for the search term and site for the site, as this is what we are going to use for the script. To make things easier, also have an ID called customsearch on the form.
To use BOSS, you should get your own developer API for BOSS and replace the one in the demo code. There is click tracking on the search results to see how successful your app is, so you should make it your own.
Adding the BOSS magic
BOSS is a REST API, meaning you can use it in any HTTP request or in a browser by simply adding the right parameters to a URL. Say for example you want to search “bbc.co.uk” for “christmas” all you need to do is open the following URL:
http://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&format=xml&appid=YOUR-APPLICATION-ID
Try it out and click it to see the results in XML. We don’t want XML though, which is why we get rid of the format=xml parameter which gives us the same information in JSON:
http://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&appid=YOUR-APPLICATION-ID
JSON makes most sense when you can send the output to a function and immediately use it. For this to happen all you need is to add a callback parameter and the JSON will be wrapped in a function call. Say for example we want to call SITESEARCH.found() when the data was retrieved we can do it this way:
http://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&callback=SITESEARCH.found&appid=YOUR-APPLICATION-ID
You can use this immediately in a script node if you want to. The following code would display the total amount of search results for the term christmas on bbc.co.uk as an alert:
<script type="text/javascript">
var SITESEARCH = {};
SITESEARCH.found = function(o){
alert(o.ysearchresponse.totalhits);
}
</script>
<script type="text/javascript" src="http://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&callback=SITESEARCH.found&appid=Kzv_lcHV34HIybw0GjVkQNnw4AEXeyJ9Rb1gCZSGxSRNrcif_HdMT9qTE1y9LdI-">
</script>
However, for our example, we need to be a bit more clever with this.
Enhancing the search form
Here’s the script that enhances a search form to show results below it.
SITESEARCH = function(){
var config = {
IDs:{
searchForm:'customsearch',
term:'term',
site:'site'
},
loading:'Loading results...',
noresults:'No results found.',
appID:'YOUR-APP-ID',
results:20
};
var form;
var out;
function init(){
if(config.appID === 'YOUR-APP-ID'){
alert('Please get a real application ID!');
} else {
form = document.getElementById(config.IDs.searchForm);
if(form){
form.onsubmit = function(){
var site = document.getElementById(config.IDs.site).value;
var term = document.getElementById(config.IDs.term).value;
if(typeof site === 'string' && typeof term === 'string'){
if(typeof out !== 'undefined'){
out.parentNode.removeChild(out);
}
out = document.createElement('p');
out.appendChild(document.createTextNode(config.loading));
form.appendChild(out);
var APIurl = 'http://boss.yahooapis.com/ysearch/web/v1/' +
term + '?callback=SITESEARCH.found&sites=' +
site + '&count=' + config.results +
'&appid=' + config.appID;
var s = document.createElement('script');
s.setAttribute('src',APIurl);
s.setAttribute('type','text/javascript');
document.getElementsByTagName('head')[0].appendChild(s);
return false;
}
};
}
}
};
function found(o){
var list = document.createElement('ul');
var results = o.ysearchresponse.resultset_web;
if(results){
var item,link,description;
for(var i=0,j=results.length;i<j;i++){
item = document.createElement('li');
link = document.createElement('a');
link.setAttribute('href',results[i].clickurl);
link.innerHTML = results[i].title;
item.appendChild(link);
description = document.createElement('p');
description.innerHTML = results[i]['abstract'];
item.appendChild(description);
list.appendChild(item);
}
} else {
list = document.createElement('p');
list.appendChild(document.createTextNode(config.noresults));
}
form.replaceChild(list,out);
out = list;
};
return{
config:config,
init:init,
found:found
};
}();
Oooohhhh scary code! Let’s go through this one bit at a time:
We start by creating a module called SITESEARCH and give it an configuration object:
SITESEARCH = function(){
var config = {
IDs:{
searchForm:'customsearch',
term:'term',
site:'site'
},
loading:'Loading results...',
appID:'YOUR-APP-ID',
results:20
}
Configuration objects are a great idea to make your code easy to change and also to override. In this case you can define different IDs than the one agreed upon earlier, define a message to show when the results are loading, when there aren’t any results, the application ID and the number of results that should be displayed.
Note: you need to replace “YOUR-APP-ID” with the real ID you retrieved from BOSS, otherwise the script will complain!
var form;
var out;
function init(){
if(config.appID === 'YOUR-APP-ID'){
alert('Please get a real application ID!');
} else {
We define form and out as variables to make sure that all the methods in the module have access to them. We then check if there was a real application ID defined. If there wasn’t, the script complains and that’s that.
form = document.getElementById(config.IDs.searchForm);
if(form){
form.onsubmit = function(){
var site = document.getElementById(config.IDs.site).value;
var term = document.getElementById(config.IDs.term).value;
if(typeof site === 'string' && typeof term === 'string'){
If the application ID was a winner, we check if the form with the provided ID exists and apply an onsubmit event handler. The first thing we get is the values of the site we want to search in and the term that was entered and check that those are strings.
if(typeof out !== 'undefined'){
out.parentNode.removeChild(out);
}
out = document.createElement('p');
out.appendChild(document.createTextNode(config.loading));
form.appendChild(out);
If both are strings we check of out is undefined. We will create a loading message and subsequently the list of search results later on and store them in this variable. So if out is defined, it’ll be an old version of a search (as users will re-submit the form over and over again) and we need to remove that old version.
We then create a paragraph with the loading message and append it to the form.
var APIurl = 'http://boss.yahooapis.com/ysearch/web/v1/' +
term + '?callback=SITESEARCH.found&sites=' +
site + '&count=' + config.results +
'&appid=' + config.appID;
var s = document.createElement('script');
s.setAttribute('src',APIurl);
s.setAttribute('type','text/javascript');
document.getElementsByTagName('head')[0].appendChild(s);
return false;
}
};
}
}
};
Now it is time to call the BOSS API by assembling a correct REST URL, create a script node and apply it to the head of the document. We return false to ensure the form does not get submitted as we want to stay on the page.
Notice that we are using SITESEARCH.found as the callback method, which means that we need to define this one to deal with the data returned by the API.
function found(o){
var list = document.createElement('ul');
var results = o.ysearchresponse.resultset_web;
if(results){
var item,link,description;
We create a new list and then get the resultset_web array from the data returned from the API. If there aren’t any results returned, this array will not exist which is why we need to check for it. Once we done that we can define three variables to repeatedly store the item title we want to display, the link to point to and the description of the link.
for(var i=0,j=results.length;i<j;i++){
item = document.createElement('li');
link = document.createElement('a');
link.setAttribute('href',results[i].clickurl);
link.innerHTML = results[i].title;
item.appendChild(link);
description = document.createElement('p');
description.innerHTML = results[i]['abstract'];
item.appendChild(description);
list.appendChild(item);
}
We then loop over the results array and assemble a list of results with the titles in links and paragraphs with the abstract of the site. Notice the bracket notation for abstract as abstract is a reserved word in JavaScript2 :).
} else {
list = document.createElement('p');
list.appendChild(document.createTextNode(config.noresults));
}
form.replaceChild(list,out);
out = list;
};
If there aren’t any results, we define a paragraph with the no results message as list. In any case we replace the old out (the loading message) with the list and re-define out as the list.
return{
config:config,
init:init,
found:found
};
}();
All that is left to do is return the properties and methods we want to make public. In this case found needs to be public as it is accessed by the API return. We return init to make it accessible and config to allow implementers to override any of the properties.
Using the script
In order to use this script, all you need to do is to add it after the form in the document, override the API key with your own and call init():
<form id="customsearch" action="http://search.yahoo.com/search">
<div>
<label for="p">Search this site:</label>
<input type="text" name="p" id="term">
<input type="hidden" name="vs" id="site" value="bbc.co.uk">
<input type="submit" value="go">
</div>
</form>
<script type="text/javascript" src="boss-site-search.js"></script>
<script type="text/javascript">
SITESEARCH.config.appID = 'copy-the-id-you-know-to-get-where';
SITESEARCH.init();
</script>
Where to go from here
This is just a very simple example of what you can do with BOSS. You can define languages and regions, retrieve and display images and news and mix the results with other data sources before displaying them. One very cool feature is that by adding a view=keyterms parameter to the URL you can get the keywords of each of the results to drill deeper into the search. An example for this written in PHP is available on the YDN blog. For JavaScript solutions there is a handy wrapper called yboss available to help you go nuts. |
2008 |
Christian Heilmann |
chrisheilmann |
2008-12-04T00:00:00+00:00 |
https://24ways.org/2008/sitewide-search-on-a-shoestring/ |
code |
| 106 |
Checking Out: Progress Meters |
It’s the holiday season, so you know what that means: online shopping! When I started developing Web sites back in the 90s, many of my first clients were small local shops wanting to sell their goods online, so I developed many a checkout system. And because of slow dial-up speeds back then, informing the user about where they were in the checkout process was pretty important.
Even though we’re (mostly) beyond the dial-up days, informing users about where they are in a flow is still important. In usability tests at the companies I’ve worked at, I’ve seen time and time again how not adequately informing the user about their state can cause real frustration. This is especially true for two sets of users: mobile users and users of assistive devices, in particular, screen readers.
The progress meter is a very common design solution used to indicate to the user’s state within a flow. On the design side, much effort may go in to crafting a solution that is as visually informative as possible. On the development side, however, solutions range widely. I’ve checked out the checkouts at a number of sites and here’s what I’ve found when it comes to progress meters: they’re sometimes inaccessible and often confusing or unhelpful — all because of the way in which they’re coded. For those who use assistive devices or text-only browsers, there must be a better way to code the progress meter — and there is.
(Note: All code samples are from live sites but have been tweaked to hide the culprits’ identities.)
How not to make progress
A number of sites assemble their progress meters using non- or semi-semantic markup and images with no alternate text. On text-only browsers (like my mobile phone) and to screen readers, this looks and reads like chunks of content with no context given.
<div id="progress">
<img src="icon_progress_1a.gif" alt="">
<em>Shipping information</em>
<img src="icon_progress_arrow.gif" alt="">
<img src="icon_progress_2a.gif" alt="">
<em>Payment information</em>
<img src="icon_progress_arrow.gif" alt="" class="progarrow">
<img src="icon_progress_3b.gif" alt="">
<strong>Place your order</strong>
</div>
In the above example, the third state, “Place your order”, is the current state. But a screen reader may not know that, and my cell phone only displays "Shipping informationPayment informationPlace your order". Not good.
Is this progress?
Other sites present the entire progress meter as a graphic, like the following:
Now, I have no problem with using a graphic to render a very stylish progress meter (my sample above is probably not the most stylish example, of course, but you understand my point). What becomes important in this case is the use of appropriate alternate text to describe the image. Disappointingly, sites today have a wide range of solutions, including using no alternate text. Check out these code samples which call progress meter images.
<img src="checkout_step2.gif" alt="">
I think we can all agree that the above is bad, unless you really don’t care whether or not users know where they are in a flow.
<img src="checkout_step2.gif" alt="Shipping information - Payment information - Place your order">
The alt text in the example above just copies all of the text found in the graphic, but it doesn’t represent the status at all. So for every page in the checkout, the user sees or hears the same text. Sure, by the second or third page in the flow, the user has figured out what’s going on, but she or he had to think about it. I don’t think that’s good.
<img src="checkout_step2.gif" alt="Checkout: Payment information">
The above probably has the best alternate text out of these examples, because the user at least understands that they’re in the Checkout process, on the Place your order page. But going through the flow with alt text like this, the user doesn’t know how many steps are in the flow.
Semantic progress
Of course, there are some sites that use an ordered list when marking up the progress meter. Hooray! Unfortunately, no text-only browser or screen reader would be able to describe the user’s current state given this markup.
<ol class="progressmeter">
<li class="one current">shipping information</li>
<li class="two">payment information</li>
<li class="three">place your order</li>
</ol>
Without CSS enabled, the above is rendered as follows:
Progress at last
We all know that semantic markup makes for the best foundation, so we’ll start with the markup found above. In order to make the state information accessible, let’s add some additional text in paragraph and span elements.
<div class="progressmeter">
<p>There are three steps in this checkout process.</p>
<ol>
<li class="one"><span>Enter your</span> shipping information</li>
<li class="two"><span>Enter your</span> payment information</li>
<li class="three"><span>Review details and</span> place your order</li>
</ol>
</div>
Add on some simple CSS to hide the paragraph and spans, and arrange the list items on a single line with a background image to represent the large number, and this is what you’ll get:
There are three steps in this checkout process.
Enter your shipping information
Enter your payment information
Review details and place your order
To display and describe a state as active, add the class “current” to one of the list items. Then change the hidden content such that it better describes the state to the user.
<div class="progressmeter">
<p>There are three steps in this checkout process.</p>
<ol>
<li class="one current"><span>You are currently entering your</span> shipping information</li>
<li class="two"><span>In the next step, you will enter your</span> payment information</li>
<li class="three"><span>In the last step, you will review the details and</span> place your order</li>
</ol>
</div>
The end result is an attractive progress meter that gives much greater semantic and contextual information.
There are three steps in this checkout process.
You are currently entering your shipping information
In the next step, you will enter your payment information
In the last step, you will review the details and place your order
For example, the above example renders in a text-only browser as follows:
There are three steps in this checkout process.
You are currently entering your shipping information
In the next step, you will enter your payment information
In the last step, you will review the details and place your order
And the screen reader I use for testing announces the following:
There are three steps in this checkout process. List of three items. You are currently entering your shipping information. In the next step, you will enter your payment information. In the last step, you will review the details and place your order. List end.
Here’s a sample code page that summarises this approach.
Happy frustration-free online shopping with this improved progress meter! |
2008 |
Kimberly Blessing |
kimberlyblessing |
2008-12-12T00:00:00+00:00 |
https://24ways.org/2008/checking-out-progress-meters/ |
ux |
| 107 |
Using Google App Engine as Your Own Content Delivery Network |
Do you remember, years ago, when hosting was expensive, domain names were the province of the rich, and you hosted your web pages on Geocities? It seems odd to me now that there was a time when each and every geek didn’t have his own top-level domain and super hosting setup. But as the parts became more and more affordable a man could become an outcast if he didn’t have his own slightly surreal-sounding TLD.
And so it will be in the future when people realise with surprise there was a time before affordable content delivery networks.
A content delivery network, or CDN, is a system of servers spread around the world, serving files from the nearest physical location. Instead of waiting for a file to find its way from a server farm in Silicon Valley 8,000 kilometres away, I can receive it from London, Dublin, or Paris, cutting down the time I wait. The big names — Google, Yahoo, Amazon, et al — use CDNs for their sites, but they’ve always been far too expensive for us mere mortals. Until now.
There’s a service out there ready for you to use as your very own CDN. You have the company’s blessing, you won’t need to write a line of code, and — best of all — it’s free. The name? Google App Engine.
In this article you’ll find out how to set up a CDN on Google App Engine. You’ll get the development software running on your own computer, tell App Engine what files to serve, upload them to a web site, and give everyone round the world access to them.
Creating your first Google App Engine project
Before we do anything else, you’ll need to download the Google App Engine software development kit (SDK). You’ll need Python 2.5 too — you won’t be writing any Python code but the App Engine SDK will need it to run on your computer. If you don’t have Python, App Engine will install it for you (if you use Mac OS X 10.5 or a Linux-based OS you’ll have Python; if you use Windows you won’t).
Done that? Excellent, because that’s the hardest step. The rest is plain sailing.
You’ll need to choose a unique ‘application id’ — nothing more than a name — for your project. Make sure it consists only of lowercase letters and numbers. For this article I’ll use 24ways2008, but you can choose anything you like.
On your computer, create a folder named after your application id. This folder can be anywhere you want: your desktop, your documents folder, or wherever you usually keep your web files. Within your new folder, create a folder called assets, and within that folder create three folders called images, css, and javascript. These three folders are the ones you’ll fill with files and serve from your content delivery network. You can have other folders too, if you like.
That will leave you with a folder structure like this:
24ways2008/
assets/
css/
images/
javascript/
Now you need to put a few files in these folders, so we can later see our CDN in action. You can put anything you want in these folders, but for this example we’ll include an HTML file, a style sheet, an image, and a Javascript library.
In the top-level folder (the one I’ve called 24ways2008), create a file called index.html. Fill this with any content you want. In the assets/css folder, create a file named core.css and throw in a couple of CSS rules for good measure. In the assets/images directory save any image that takes your fancy — I’ve used the silver badge from the App Engine download page. Finally, to fill the JavaScript folder, add in this jQuery library file. If you’ve got the time and the inclination, you can build a page that uses all these elements.
So now we should have a set of files and folders that look something like this:
24ways2008/
assets/
index.html
css/
core.css
images/
appengine-silver-120x30.gif
javascript/
jquery-1.2.6.min.js
Which leaves us with one last file to create. This is the important one: it tells App Engine what to do with your files. It’s named app.yaml, it sits at the top-level (inside the folder I’ve named 24ways2008), and it needs to include these lines:
application: 24ways2008
version: 1
runtime: python
api_version: 1
handlers:
- url: /
static_files: assets/index.html
upload: assets/index.html
- url: /
static_dir: assets
You need to make sure you change 24ways2008 on the first line to whatever you chose as your application id, but otherwise the content of your app.yaml file should be identical. And with that, you’ve created your first App Engine project. If you want it, you can download a zip file containing my project.
Testing your project
As it stands, your project is ready to be uploaded to App Engine. But we couldn’t call ourselves professionals if we didn’t test it, could we? So, let’s put that downloaded SDK to good use and run the project from your own computer.
One of the files you’ll find App Engine installed is named dev_appserver.py, a Python script used to simulate App Engine on your computer. You’ll find lots of information on how to do this in the documentation on the development web server, but it boils down to running the script like so (the space and the dot at the end are important):
dev_appserver.py .
You’ll need to run this from the command-line: Mac users can run the Terminal application, Linux users can run their favourite shell, and Windows users will need to run it via the Command Prompt (open the Start menu, choose ‘Run…’, type ‘cmd‘, and click ‘OK’). Before you run the script you’ll need to make sure you’re in the project folder — in my case, as I saved it to my desktop I can go there by typing
cd ~/Desktop/24ways2008
in my Mac’s Terminal app; if you’re using Windows you can type
cd "C:\Documents and Settings\username\Desktop\24ways2008"
If that’s successful, you’ll see a few lines of output, the last looking something like this:
INFO 2008-11-22 14:35:00,830 dev_appserver_main.py] Running application 24ways2008 on port 8080: http://localhost:8080
Now you can power up your favourite browser, point it to http://localhost:8080/, and you’ll see the page you saved as index.html. You’ll also find your CSS file at http://localhost:8080/css/core.css. In fact, anything you put inside the assets folder in the project will be accessible from this domain. You’re running our own App Engine web server!
Note that no-one else will be able to see your files: localhost is a special domain that you can only see from your computer — and once you stop the development server (by pressing Control–C) you’ll not be able to see the files in your browser until you start it again.
You might notice a new file has turned up in your project: index.yaml. App Engine creates this file when you run the development server, and it’s for internal App Engine use only. If you delete it there are no ill effects, but it will reappear when you next run the development server. If you’re using version control (e.g. Subversion) there’s no need to keep a copy in your repository.
So you’ve tested your project and you’ve seen it working on your own machine; now all you need to do is upload your project and the world will be able to see your files too.
Uploading your project
If you don’t have a Google account, create one and then sign in to App Engine. Tell Google about your new project by clicking on the ‘Create an Application’ button. Enter your application id, give the application a name, and agree to the terms and conditions. That’s it. All we need do now is upload the files.
Open your Mac OS X Terminal, Windows Command Prompt, or Linux shell window again, move to the project folder, and type (again, the space and the dot at the end are important):
appcfg.py update .
Enter your email address and password when prompted, and let App Engine do it’s thing. It’ll take no more than a few seconds, but in that time App Engine will have done the equivalent of logging in to an FTP server and copying files across. It’s fairly understated, but you now have your own project up and running. You can see mine at http://24ways2008.appspot.com/, and everyone can see yours at http://your-application-id.appspot.com/. Your files are being served up over Google’s content delivery network, at no cost to you!
Benefits of using Google App Engine
The benefits of App Engine as a CDN are obvious: your own server doesn’t suck up the bandwidth, while your visitors will appreciate a faster site. But there are also less obvious benefits.
First, once you’ve set up your site, updating it is an absolute breeze. Each time you update a file (or a batch of files) you need only run appcfg.py to see the changes appear on your site. To paraphrase Joel Spolsky, a good web site must be able to be updated in a single step. Many designers and developers can’t make that claim, but with App Engine, you can.
App Engine also allows multiple people to work on one application. If you want a friend to be able to upload files to your site you can let him do so without giving him usernames and passwords — all he needs is his own Google account. App Engine also gives you a log of all actions taken by collaborators, so you can see who’s made updates, and when.
Another bonus is the simple version control App Engine offers. Do you remember the file named app.yaml you created a while back? The second line looked like this:
version: 1
If you change the version number to 2 (or 3, or 4, etc), App Engine will keep a copy of the last version you uploaded. If anything goes wrong with your latest version, you can tell App Engine to revert back to that last saved version. It’s no proper version control system, but it could get you out of a sticky situation.
One last thing to note: if you’re not happy using your-application-id.appspot.com as your domain, App Engine will quite happily use any domain you own.
The weak points of Google App Engine
In the right circumstances, App Engine can be a real boon. I run my own site using the method I’ve discussed above, and I’m very happy with it. But App Engine does have its disadvantages, most notably those discussed by Aral Balkan in his post ‘Why Google App Engine is broken and what Google must do to fix it‘.
Aral found the biggest problems while using App Engine as a web application platform; I wouldn’t recommend using it as such either (at least for now) but for our purposes as a CDN for static files, it’s much more worthy. Still, App Engine has two shortcomings you should be aware of.
The first is that you can’t host a file larger than one megabyte. If you want to use App Engine to host that 4.3MB download for your latest-and-greatest desktop software, you’re out of luck. The only solution is to stick to smaller files.
The second problem is the quota system. Google’s own documentation says you’re allowed 650,000 requests a day and 10,000 megabytes of bandwidth in and out (20,000 megabytes in total), which should be plenty for most sites. But people have seen sites shut down temporarily for breaching quotas — in some cases after inexplicable jumps in Google’s server CPU usage. Aral, who’s seen it happen to his own sites, seemed genuinely frustrated by this, and if you measure your hits in the hundreds of thousands and don’t want to worry about uptime, App Engine isn’t for you.
That said, for most of us, App Engine offers a fantastic resource: the ability to host files on Google’s own content delivery network, at no charge.
Conclusion
If you’ve come this far, you’ve seen how to create a Google App Engine project and host your own files on Google’s CDN. You’ve seen the great advantages App Engine offers — an excellent content delivery network, the ability to update your site with a single command, multiple authors, simple version control, and the use of your own domain — and you’ve come across some of its weaknesses — most importantly the limit on file sizes and the quota system. All that’s left to do is upload those applications — but not before you’ve finished your Christmas shopping. |
2008 |
Matt Riggott |
mattriggott |
2008-12-06T00:00:00+00:00 |
https://24ways.org/2008/using-google-app-engine-as-your-own-cdn/ |
process |
| 109 |
Geotag Everywhere with Fire Eagle |
A note from the editors: Since this article was written Yahoo! has retired the Fire Eagle service.
Location, they say, is everywhere. Everyone has one, all of the time. But on the web, it’s taken until this year to see the emergence of location in the applications we use and build.
The possibilities are broad. Increasingly, mobile phones provide SDKs to approximate your location wherever you are, browser extensions such as Loki and Mozilla’s Geode provide browser-level APIs to establish your location from the proximity of wireless networks to your laptop. Yahoo’s Brickhouse group launched Fire Eagle, an ambitious location broker enabling people to take their location from any of these devices or sources, and provide it to a plethora of web services. It enables you to take the location information that only your iPhone knows about and use it anywhere on the web.
That said, this is still a time of location as an emerging technology. Fire Eagle stores your location on the web (protected by application-specific access controls), but to try and give an idea of how useful and powerful your location can be — regardless of the services you use now — today’s 24ways is going to build a bookmarklet to call up your location on demand, in any web application.
Location Support on the Web
Over the past year, the number of applications implementing location features has increased dramatically. Plazes and Brightkite are both full featured social networks based around where you are, whilst Pownce rolled in Fire Eagle support to allow geotagging of all the content you post to their microblogging service. Dipity’s beautiful timeline shows for you moving from place to place and Six Apart’s activity stream for Movable Type started exposing your movements.
The number of services that hook into Fire Eagle will increase as location awareness spreads through the developer community, but you can use your location on other sites indirectly too.
Consider Flickr. Now world renowned for their incredible mapping and places features, geotagging on Flickr started out as a grassroots extension of regular tagging. That same technique can be used to start rolling geotagging in any publishing platform you come across, for any kind of content. Machine-tags (geo:lat= and geo:lon=) and the adr and geo microformats can be used to enhance anything you write with location information.
A crash course in avian inflammability
Fire Eagle is a location store. A broker between services and devices which provide location and those which consume it. It’s a switchboard that controls which pieces of your location different applications can see and use, and keeps hidden anything you want kept private. A blog widget that displays your current location in public can be restricted to display just your current city, whilst a service that provides you with a list of the nearest ATMs will operate better with a precise street address.
Even if your iPhone tells Fire Eagle exactly where you are, consuming applications only see what you want them to see. That’s important for users to realise that they’re in control, but also important for application developers to remember that you cannot rely on having super-accurate information available all the time. You need to build location aware applications which degrade gracefully, because users will provide fuzzier information — either through choice, or through less accurate sources.
Application specific permissions are controlled through an OAuth API. Each application has a unique key, used to request a second, user-specific key that permits access to that user’s information. You store that user key and it remains valid until such a time as the user revokes your application’s access. Unlike with passwords, these keys are unique per application, so revoking the access rights of one application doesn’t break all the others.
Building your first Fire Eagle app; Geomarklet
Fire Eagle’s developer documentation can take you through examples of writing simple applications using server side technologies (PHP, Python). Here, we’re going to write a client-side bookmarklet to make your location available in every site you use. It’s designed to fast-track the experience of having location available everywhere on web, and show you how that can be really handy. Hopefully, this will set you thinking about how location can enhance the new applications you build in 2009.
An oddity of bookmarklets
Bookmarklets (or ‘favlets’, for those of an MSIE persuasion) are a strange environment to program in. Critically, you have no persistent storage available. As such, using token-auth APIs in a static environment requires you to build you application in a slightly strange way; authing yourself in advance and then hardcoding the keys into your script.
Get started
Before you do anything else, go to http://fireeagle.com and log in, get set up if you need to and by all means take a look around. Take a look at the mobile updaters section of the application gallery and perhaps pick out an app that will update Fire Eagle from your phone or laptop.
Once that’s done, you need to register for an application key in the developer section. Head straight to /developer/create and complete the form. Since you’re building a standalone application, choose ‘Auth for desktop applications’ (rather than web applications), and select that you’ll be ‘accessing location’, not updating.
At the end of this process, you’ll have two application keys, a ‘Consumer Key’ and a ‘Consumer Secret’, which look like these:
Consumer Key
luKrM9U1pMnu
Consumer Secret
ZZl9YXXoJX5KLiKyVrMZffNEaBnxnd6M
These keys combined allow your application to make requests to Fire Eagle.
Next up, you need to auth yourself; granting your new application permission to use your location. Because bookmarklets don’t have local storage, you can’t integrate the auth process into the bookmarklet itself — it would have no way of storing the returned key. Instead, I’ve put together a simple web frontend through which you can auth with your application.
Head to Auth me, Amadeus!, enter the application keys you just generated and hit ‘Authorize with Fire Eagle’. You’ll be taken to the Fire Eagle website, just as in regular Fire Eagle applications, and after granting access to your app, be redirected back to Amadeus which will provide you your user tokens. These tokens are used in subsequent requests to read your location.
And, skip to the end…
The process of building the bookmarklet, making requests to Fire Eagle, rendering it to the page and so forth follows, but if you’re the impatient type, you might like to try this out right now. Take your four API keys from above, and drag the following link to your Bookmarks Toolbar; it contains all the code described below. Before you can use it, you need to edit in your own API keys. Open your browser’s bookmark editor and where you find text like ‘YOUR_CONSUMER_KEY_HERE’, swap in the corresponding key you just generated.
Get Location
Bookmarklet Basics
To start on the bookmarklet code, set out a basic JavaScript module-pattern structure:
var Geomarklet = function() {
return ({
callback: function(json) {},
run: function() {}
});
};
Geomarklet.run();
Next we’ll add the keys obtained in the setup step, and also some basic Fire Eagle support objects:
var Geomarklet = function() {
var Keys = {
consumer_key: 'IuKrJUHU1pMnu',
consumer_secret: 'ZZl9YXXoJX5KLiKyVEERTfNEaBnxnd6M',
user_token: 'xxxxxxxxxxxx',
user_secret: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
};
var LocationDetail = {
EXACT: 0,
POSTAL: 1,
NEIGHBORHOOD: 2,
CITY: 3,
REGION: 4,
STATE: 5,
COUNTRY: 6
};
var index_offset;
return ({
callback: function(json) {},
run: function() {}
});
};
Geomarklet.run();
The Location Hierarchy
A successful Fire Eagle query returns an object called the ‘location hierarchy’. Depending on the level of detail shared, the index of a particular piece of information in the array will vary. The LocationDetail object maps the array indices of each level in the hierarchy to something comprehensible, whilst the index_offset variable is an adjustment based on the detail of the result returned.
The location hierarchy object looks like this, providing a granular breakdown of a location, in human consumable and machine-friendly forms.
"user": {
"location_hierarchy": [{
"level": 0,
"level_name": "exact",
"name": "707 19th St, San Francisco, CA",
"normal_name": "94123",
"geometry": {
"type": "Point",
"coordinates": [ - 0.2347530752, 67.232323]
},
"label": null,
"best_guess": true,
"id": ,
"located_at": "2008-12-18T00:49:58-08:00",
"query": "q=707%2019th%20Street,%20Sf"
},
{
"level": 1,
"level_name": "postal",
"name": "San Francisco, CA 94114",
"normal_name": "12345",
"woeid": ,
"place_id": "",
"geometry": {
"type": "Polygon",
"coordinates": [],
"bbox": []
},
"label": null,
"best_guess": false,
"id": 59358791,
"located_at": "2008-12-18T00:49:58-08:00"
},
{
"level": 2,
"level_name": "neighborhood",
"name": "The Mission, San Francisco, CA",
"normal_name": "The Mission",
"woeid": 23512048,
"place_id": "Y12JWsKbApmnSQpbQg",
"geometry": {
"type": "Polygon",
"coordinates": [],
"bbox": []
},
"label": null,
"best_guess": false,
"id": 59358801,
"located_at": "2008-12-18T00:49:58-08:00"
},
}
In this case the first object has a level of 0, so the index_offset is also 0.
Prerequisites
To query Fire Eagle we call in some existing libraries to handle the OAuth layer and the Fire Eagle API call. Your bookmarklet will need to add the following scripts into the page:
The SHA1 encryption algorithm
The OAuth wrapper
An extension for the OAuth wrapper
The Fire Eagle wrapper itself
When the bookmarklet is first run, we’ll insert these scripts into the document. We’re also inserting a stylesheet to dress up the UI that will be generated.
If you want to follow along any of the more mundane parts of the bookmarklet, you can download the full source code.
Rendering
This bookmarklet can be extended to support any formatting of your location you like, but for sake of example I’m going to build three common formatters that you’ll find useful for common location scenarios: Sites which already ask for your location; and in publishing systems that accept tags or HTML mark-up.
All the rendering functions are items in a renderers object, so they can be iterated through easily, making it trivial to add new formatting functions as your find new use cases (just add another function to the object).
var renderers = {
geotag: function(user) {
if(LocationDetail.EXACT !== index_offset) {
return false;
}
else {
var coords =
user.location_hierarchy[LocationDetail.EXACT].geometry.coordinates;
return "geo:lat=" + coords[0] + ", geo:lon=" + coords[1];
}
},
city: function(user) {
if(LocationDetail.CITY < index_offset) {
return false;
}
else {
return user.location_hierarchy[LocationDetail.CITY - index_offset].name;
}
}
You should always fail gracefully, and in line with catering to users who choose not to share their location precisely, always check that the location has been returned at the level you require. Geotags are expected to be precise, so if an exact location is unavailable, returning false will tell the rendering aspect of the bookmarklet to ignore the function altogether.
These first two are quite simple, geotag returns geo:lat=-0.2347530752, geo:lon=67.232323 and city returns San Francisco, CA.
This final renderer creates a chunk of HTML using the adr and geo microformats, using all available aspects of the location hierarchy, and can be used to geotag any content you write on your blog or in comments:
html: function(user) {
var geostring = '';
var adrstring = '';
var adr = [];
adr.push('<p class="adr">');
// city
if(LocationDetail.CITY >= index_offset) {
adr.push(
'\n <span class="locality">'
+ user.location_hierarchy[LocationDetail.CITY-index_offset].normal_name
+ '</span>,'
);
}
// county
if(LocationDetail.REGION >= index_offset) {
adr.push(
'\n <span class="region">'
+ user.location_hierarchy[LocationDetail.REGION-index_offset].normal_name
+ '</span>,'
);
}
// locality
if(LocationDetail.STATE >= index_offset) {
adr.push(
'\n <span class="region">'
+ user.location_hierarchy[LocationDetail.STATE-index_offset].normal_name
+ '</span>,'
);
}
// country
if(LocationDetail.COUNTRY >= index_offset) {
adr.push(
'\n <span class="country-name">'
+ user.location_hierarchy[LocationDetail.COUNTRY-index_offset].normal_name
+ '</span>'
);
}
// postal
if(LocationDetail.POSTAL >= index_offset) {
adr.push(
'\n <span class="postal-code">'
+ user.location_hierarchy[LocationDetail.POSTAL-index_offset].normal_name
+ '</span>,'
);
}
adr.push('\n</p>\n');
adrstring = adr.join('');
if(LocationDetail.EXACT === index_offset) {
var coords =
user.location_hierarchy[LocationDetail.EXACT].geometry.coordinates;
geostring = '<p class="geo">'
+'\n <span class="latitude">'
+ coords[0]
+ '</span>;'
+ '\n <span class="longitude">'
+ coords[1]
+ '</span>\n</p>\n';
}
return (adrstring + geostring);
}
Here we check the availability of every level of location and build it into the adr and geo patterns as appropriate. Just as for the geotag function, if there’s no exact location the geo markup won’t be returned.
Finally, there’s a rendering method which creates a container for all this data, renders all the applicable location formats and then displays them in the page for a user to copy and paste. You can throw this together with DOM methods and some simple styling, or roll in some components from YUI or JQuery to handle drawing full featured overlays.
You can see this simple implementation for rendering in the full source code.
Make the call
With a framework in place to render Fire Eagle’s location hierarchy, the only thing that remains is to actually request your location. Having already authed through Amadeus earlier, that’s as simple as instantiating the Fire Eagle JavaScript wrapper and making a single function call. It’s a big deal that whilst a lot of new technologies like OAuth add some complexity and require new knowledge to work with, APIs like Fire Eagle are really very simple indeed.
return {
run: function() {
insert_prerequisites();
setTimeout(
function() {
var fe = new FireEagle(
Keys.consumer_key,
Keys.consumer_secret,
Keys.user_token,
Keys.user_secret
);
var script = document.createElement('script');
script.type = 'text/javascript';
script.src = fe.getUserUrl(
FireEagle.RESPONSE_FORMAT.json,
'Geomarklet.callback'
);
document.body.appendChild(script);
},
2000
);
},
callback: function(json) {
if(json.rsp && 'fail' == json.rsp.stat) {
alert('Error ' + json.rsp.code + ": " + json.rsp.message);
}
else {
index_offset = json.user.location_hierarchy[0].level;
draw_selector(json);
}
}
};
We first insert the prerequisite scripts required for the Fire Eagle request to function, and to prevent trying to instantiate the FireEagle object before it’s been loaded over the wire, the remaining instantiation and request is wrapped inside a setTimeout delay.
We then create the request URL, referencing the Geomarklet.callback callback function and then append the script to the document body — allowing a cross-domain request.
The callback itself is quite simple. Check for the presence and value of rsp.status to test for errors, and display them as required. If the request is successful set the index_offset — to adjust for the granularity of the location hierarchy — and then pass the object to the renderer.
The result? When Geomarklet.run() is called, your location from Fire Eagle is read, and each renderer displayed on the page in an easily copy and pasteable form, ready to be used however you need.
Deploy
The final step is to convert this code into a long string for use as a bookmarklet. Easiest for Mac users is the JavaScript bundle in TextMate — choose Bundles: JavaScript: Copy as Bookmarklet to Clipboard. Then create a new ‘Get Location’ bookmark in your browser of choice and paste in.
Those without TextMate can shrink their code down into a single line by first running their code through the JSLint tool (to ensure the code is free from errors and has all the required semi-colons) and then use a find-and-replace tool to remove line breaks from your code (or even run your code through JSMin to shrink it down).
With the bookmarklet created and added to your bookmarks bar, you can now call up your location on any page at all. Get a feel for a web where your location is just another reliable part of the browsing experience.
Where next?
So, the Geomarklet you’ve been guided through is a pretty simple premise and pretty simple output. But from this base you can start to extend: Add code that will insert each of the location renderings directly into form fields, perhaps, or how about site-specific handlers to add your location tags into the correct form field in Wordpress or Tumblr? Paste in your current location to Google Maps? Or Flickr?
Geomarklet gives you a base to start experimenting with location on your own pages and the sites you browse daily.
The introduction of consumer accessible geo to the web is an adventure of discovery; not so much discovering new locations, but discovering location itself. |
2008 |
Ben Ward |
benward |
2008-12-21T00:00:00+00:00 |
https://24ways.org/2008/geotag-everywhere-with-fire-eagle/ |
code |