tag:blogger.com,1999:blog-85063025961392795372024-02-07T00:40:59.294-05:00The Sleepless GeekMy old blog - <a href="http://nathanmlong.com/blog">come see my new one</a>Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comBlogger39125tag:blogger.com,1999:blog-8506302596139279537.post-18477598309090603232012-07-03T10:35:00.001-04:002012-07-03T10:35:21.176-04:00Notification from long-running tasksYesterday I thought of a technique which has become a real time-saver for me. (I'm sure I'm not the first person to think of this; I remember reading something similar in Time Management for System Administrators.)<br />
<br />
You can get notification from any long-running task on the command line like this:<br />
<br />
<pre class="code">
do_my_task && make_a_sound<br />
</pre>
<br />
For example, using Rake on a Mac:<br />
<br />
<pre class="code">
rake db:reseed && say 'done'<br />
</pre>
<br />
Now I can kick of the database to be rebuilt and move to another window, knowing that I'll get audible notification when that task is complete.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-88901411976045680962012-06-18T13:38:00.002-04:002012-06-18T13:39:42.394-04:00HubbubI like to imagine that somewhere in Silicon Valley, there is a sandwich shop which caters to nerds. They send out special deals via RSS feed. Naturally, they call this the PubsubHubbub Sub Club.<br />
<br />
Based on the special they're running, they keep extra ingredients handy to satisfy demand. Each morning, they must fill the PubsubHubbub Sub Club Grub Tub.<br />
<br />
At the end of the day, they clean it out and wash it thoroughly. They do this, of course, using the PubsubHubbub Sub Club Grub Tub Scrub Nub.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-72002996066512905132012-05-24T12:10:00.000-04:002012-05-24T12:10:50.139-04:00Programming music: Afriki Djigui TheatriI've been meaning to write a post about music I like to listen to while programming. My main criteria is that it doesn't have words I can understand, because that triggers the language part of my brain, which makes it impossible for me to decide what to call my variable.<br />
<br />
A full post will have to wait, but the moment, I'm <b>really</b> enjoying a radio station full of African music. I found it in iTunes: Radio -> International/World > Afriki Djigui Theatri. They also have a website: http://www.djigui.org/<br />
<br />
It's beautiful, most of it has a nice beat, and I can't understand a word.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-69297936480117816312012-02-01T13:59:00.001-05:002012-02-01T13:59:12.242-05:00The beauty of xargsEvery time we deploy our Rails application, our deployment tool creates a new folder based on the date and time, puts the right files in it, and when everything is ready to go, updates a symlink to point to the newest release.<br/><br/>
Yesterday I found that I couldn't deploy because we were out of disk space. I needed to delete some of the old release folders.<br/><br/>
When I sshed into the server and looked in the releases folder, it had a ton of folders named like this:<br/>
<pre class="code">20120126191222
20120126193901
20120127153732
20120127171244
20120127204235
20120127204517
20120130172837
20120131152908
20120131160422
</pre>
Here you see some from 2012, but they went back all the way to 2010. I wanted to delete everything older than 2012.<br/><br/>
Enter the magic of <strong>xargs</strong>. It lets you take the output of one command and input each line of it as an argument to another command.<br/><br/>
Here's what I did:
<pre class="code">ls # shows me all the folders
ls | grep ^201[01] # shows only the ones starting with 2010 or 2011
ls | grep ^201[01] | xargs rm -rf # delete all those
</pre>
Building up the command bit-by-bit lets me verify that I'm going to delete the right things. And xargs knocks it out.<br/><br/>
Here's another useful example. Vim creates a temporary .swp files that sometimes don't get cleaned up. To find and delete them all out of a folder:<br/>
<pre class="code">find . -name '*.swp' | xargs rm
</pre>Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-92232602031213168752011-12-22T06:25:00.000-05:002011-12-22T06:25:00.658-05:00Faster feedbackIn programming, the faster you get feedback from your code, the more productive you can be.<br />
<br />
For example, suppose you're going to use the Javascript <span class="code">Date</span> class in a web page to do some time calculations. Do you:<br />
<ul>
<li>Write some code, save the file, reload the page, look at the results, and repeat?</li>
<li>Open a Javascript console (like Chome Developer Tools or Firebug) and experiment?</li>
</ul>
The second method gives you instant feedback. This means you can find the right approach more quickly and not lose focus in the meantime. The same is true when coding Ruby: if you want to see what <span class="code">Array#select</span> does, irb will give you much faster feedback than a Rails app.<br />
<br />
Other ways you can get feedback faster include:<br />
<ul>
<li>To learn more about Git, create a temporary folder and a dummy project: <span class="code">mkdir ~/test; cd ~/test; git init; echo 'hello' > test.txt; git add .; git commit -m"first";</span> Now you can experiment: branch and merge, rebase, and whatever else you want to try without fear of screwing up a real project. When you're satisfied, just delete that folder.</li>
<li>If you're thinking of adding a command to one of your runtime configuration files, like .bashrc or .vimrc, run the command directly first and see what it does.</li>
<li>Take the time to make your automated tests run faster: you'll be more likely to run them often and less likely to lose focus when you do.</li>
</ul>Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-82201591769728771892011-08-26T14:47:00.003-04:002011-08-26T16:40:02.543-04:00Testing a Rails 3.1 Engine's Javascript with JasmineAt work, we've got a Rails engine that provides some complex drop-in forms to various applications. The forms have associated Javascript which, on Rails 3.0, we had to copy and paste between applications. That's both annoying and fragile, since things can get out of sync.<br />
<br />
To test our engine, we had created a dummy app with the Enginex gem, located in <span class="code">spec/dummy</span>. <br />
<br />
In upgrading the engine to 3.1 (currently on release candidate 6), we wanted to move the associated JS into the engine, so that they would be versioned together in a single gem.<br />
<br />
That wasn't so hard, but we also needed our associated Jasmine tests to run. For that, we needed to precompile our Javascript before <span class="code">rake jasmine</span> ran so that Jasmine could load and test it.<br />
<br />
<h2>Precompile Problems</h2><br />
And that's where things got sticky. According to <a href="http://ryanbigg.com/guides/asset_pipeline.html#precompiling-assets">Ryan Biggs' documentation</a>, the default matcher for precompiling files is:<br />
<br />
<pre class="brush: ruby">[ /\w+\.(?!js|css).+/, /application.(css|js)$/ ]
</pre><br />
Basically, besides application.js and application.css, any js (or css) files with multiple dots in it, whether it's <span class="code">foo.js.coffee</span> or <span class="code">jquery.cookie.js</span>, will get compiled to its own file.<br />
<br />
We had two problems with that:<br />
<br />
<ul><li>It <strong>didn't</strong> match our engine's manifest file, because it's not named application.js. (It's not named that because in the host application, we will already have an application.js and we don't want a conflict.)</li>
<li>It <strong>did</strong> match a bunch of other JS files which didn't need to be compiled separately. For example, the manifest requires <span class="code">jquery.cookie.js</span>, so that file gets compiled and added into the manifest file. It doesn't need to <strong>also</strong> be compiled as <span class="code">jquery-aaff723a97d782d30e3fc2f0dee5d849.cookie.js</span>; we don't plan to serve it separately.</li>
</ul><br />
To solve the first problem, in Engine.rb, we added:<br />
<br />
<pre class="brush: ruby">initializer "myEngine.asset_pipeline" do |app|
app.config.assets.precompile << 'myEngine-manifest.js'
end
</pre><br />
To solve the second problem, we... did nothing. We don't care. We're letting it create extra compiled files that we don't need.<br />
<br />
We <strong>could</strong> have changed the match regex to only match files with endings like in <span class="code">.js.something</span> or <span class="code">.css.something</span>, instead of the broad rule that will also match <span class="code">.min.js</span>. But doing that in an engine might have broken a host application if it were compiling something we didn't foresee. Maybe at some point a host app will want to compile <span class="code">.csv.erb</span> files or something; we don't want to preclude that.<br />
<br />
So we chose "be unobtrusive to the host app" over "prevent unnecessary compilation".<br />
<br />
(We also could have prevented <span class="code">jquery.cookie.js</span> from being compiled separately by renaming it to <span class="code">jquery-cookie.js</span>, but we think that's annoying.)<br />
<br />
<h2>Jasmine Setup</h2><br />
Now that our <span class="code">myEngine-manifest.js</span> was being compiled to <span class="code">myEngine-manifest-20309309d9309330309390.js</span>, we needed to let Jasmine know to load that single file, containing all our compiled Javascript, rather than all the separate files it had to load previously. So, in jasmine.yml, we now have:<br />
<br />
<pre class="brush: ruby">src_files:
- spec/dummy/public/assets/myEngine-manifest-*.js
</pre><br />
The * will match whatever hash fingerprint is added to the filename.<br />
<br />
Now, before we can run <span class="code">rake jasmine</span>, we needed to make sure everything was compiled appropriately. So we created this simple Rake task in the engine's Rakefile:<br />
<br />
<pre class="brush: ruby">task :precompile_jasmine => ['assets:clean', 'assets:precompile', 'jasmine']
</pre><br />
Engine Rake tasks don't necessarily need access to Rails tasks, so to get the task above to work, we had to put the following above it so that we'd have access to those asset-related Rails tasks: <br />
<br />
<pre class="brush: ruby">load File.expand_path('../spec/dummy/Rakefile', <strong>FILE</strong>)</p></pre><br />
<h2>Turning on Sprockets</h2><br />
To get Sprockets working in our dummy application, we had to make two changes to <span class="code">spec/dummy/config/application.rb</span>:<br />
<br />
<ul><li>Below the other railties requirements, add <span class="code">require sprockets/railtie</span></li>
<li>Within the app settings block, add <span class="code">config.assets.enabled = true</span></li>
</ul><br />
You may not need to do this; it's a side-effect of the fact that we generated our engine with a version of the Enginex gem that preceded Rails 3.1.<br />
<br />
<h2>Other setup</h2><br />
<ul><li>To get Jasmine working with Rails 3.1, make sure you've got a new enough version. Jasmine 1.0.2.1 worked for us.</li>
<li>You don't want to check in all your compiled assets, so be sure to add this to your <span class="code">.gitignore</span>: <span class="code">spec/dummy/public/assets/*</span></li>
</ul><br />
<h2>Weirdness</h2><br />
Something weird about <span class="code">assets:precompile</span> makes it, and any task that runs after it, run twice. This means that `rake jasmine` requires two interrupts to shut down properly in our current setup. (In a previous attempt, it tried to run again while it was running, and got an error because it couldn't re-bind to the same port.)<br />
<br />
<h2>Addendum</h2><br />
There's no way I could have figured this out by myself, and I'm not sure that I'll be able to answer questions about it. <a href="https://github.com/adamhunter">Adam Hunter</a> and I worked on it together, and he contributed more brainpower than I did. But after all the blind alleys and frustration, I was determined to write up what we did, and got Adam's help doing so. So: good luck replicating this. :)<br />
<br />
Also, if you're looking to use Jasmine in a Rails 3.1 app with coffeescript, <a href="http://pivotallabs.com/users/jdean/blog/articles/1778-writing-and-running-jasmine-specs-with-rails-3-1-and-coffeescript">this blog post from Pivotal Labs</a> may be more helpful. We got some ideas from it, too.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-86314338004015646052011-07-14T12:43:00.001-04:002011-07-14T13:32:34.135-04:00Sometimes you can be too DRYIn a Rails app, my favorite idiom for describing what users can and cannot do goes like this:<br />
<br />
<pre class="brush: ruby"># In view code, for example...
link_to "Edit Widget", edit_widget_path if current_user.can_edit?(widget)
# which calls this method
class User
def can_edit?(resource)
resource.editable_by?(self)
end
end
# ... which in turn calls this method
class Widget
def editable_by?(user)
# whatever logic makes sense in this particular app
end
end
</pre><br />
I first saw this approach a couple of years ago, when I was new to Rails and found <a href="http://pivotallabs.com/users/nick/blog/articles/272-access-control-permissions-in-rails">Nick Kallen's post about</a> from 2007. John Nunemaker has since created the <a href="https://github.com/jnunemaker/canable/">Canable gem</a> to make this even easier.<br />
<br />
Defining these methods yourself is not that hard. But because they are all so similar, you might be tempted to use metaprogramming to shorten the code and make it more DRY.<br />
<br />
This is a rare example where sacrificing a little bit of DRY can make the code a lot more readable and maintainable: readable because your eyes don't have to bounce around as much, and maintainable because you can edit each method separately.<br />
<br />
<pre class="brush: ruby"> class User
# Metaprogramming way
{
:create => 'creatable',
:read => 'readable',
:edit => 'editable',
:delete => 'deletable',
:assign => 'assignable'
}.each do |verb, adjective|
define_method "can_#{verb}?".to_sym do |resource|
resource.send("#{adjective}_by?", self)
end
end
# Long-form way: less DRY but easier
# to understand at a glance
def can_create?(resource)
resource.creatable_by?(self)
end
def can_read?(resource)
resource.readable_by?(self)
end
def can_edit?(resource)
resource.editable_by?(self)
end
def can_delete?(resource)
resource.deletable_by?(self)
end
def can_assign?(resource)
resource.assignable_by?(self)
end
end
</pre>Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-19292972228667345372011-06-12T17:52:00.004-04:002011-06-12T17:55:28.391-04:00Metaprogramming Danger BoxIn <a href="http://sleeplessgeek.blogspot.com/2011/06/using-execute-around-pattern-in-ruby.html">my last post</a>, I showed how one can use the "execute around" pattern in Ruby to log any calls to a method, without cluttering the methods themselves with logging calls.<br />
<br />
That gave us code like this:<br />
<br />
<pre class="brush: ruby">def defuse_bomb
with_log('defuse_bomb') {puts "Snip the red wire..."}
end
</pre><br />
Our <span class="code">with_log</span> method took care of printing out some logging info before and after executing the block given to it.<br />
<br />
But as Adam pointed out in the comments, this is still kind of ugly. We don't have all the actual logging code inside of the "defuse_bomb" method, but have had to modify it to use with_log. What if we could add logging more transparently?<br />
<br />
We can! With some metaprogramming magic added to the Logging module, here's the way the DangerBox class looks now:<br />
<br />
<pre class="brush: ruby">class DangerBox
extend Logging
add_logging_to(:defuse_bomb, :throw_error)
def defuse_bomb
puts "Snip the red wire..."
end
def throw_error
puts 'I feel an error coming on...'
raise 'Success: an error!'
end
# This method's output won't be logged
def eat_poison
puts "Mmmmm, poison!! Nom Nom Nom Nom!!!!"
end
end
</pre><br />
It still works the same way as it did before:<br />
<br />
<pre class="brush: ruby">d = DangerBox.new
d.defuse_bomb
# ==== Sun Jun 12 16:55:49 -0400 2011 ====
# About to execute 'defuse_bomb'...
# Snip the red wire...
# Executed 'defuse_bomb' successfully.
d.throw_error
# ==== Sun Jun 12 16:55:49 -0400 2011 ====
# About to execute 'throw_error'...
# I feel an error coming on...
# Error while executing 'throw_error': 'Success: an error!'!
d.eat_poison
# Mmmmm, poison!! Nom Nom Nom Nom!!!!
</pre><br />
Adam, I see your "hmmmm" and raise you a "BOOYAH!" (Although I didn't write tests for my version of this, so you still win on that front.)<br />
<br />
But how the heck did we get all this logging output? Obviously, it's something to do with the line that says: <br />
<br />
<div class="code">add_logging_to(:defuse_bomb, :throw_error)</div><br />
Well, yes: that and a spiffy new Logging module that redefines our methods on the fly to use logging. Here's the code:<br />
<br />
<pre class="brush: ruby">module Logging
# Keeps track of the methods we want to log
def add_logging_to(*args)
@logged_methods ||= []
@logged_methods.concat(args)
end
# Anytime a method gets added to our class (like with "def"),
# this will execute
def method_added(method_name)
if @logged_methods.include?(method_name)
# A lock to prevent infinite recursion; since we're
# going to add a method below, we don't want that
# to make method_added fire again, which would
# make it fire again, etc...
unless @adding_with_metaprogramming
@adding_with_metaprogramming = true
stashed_method = "stashed_#{method_name}".to_sym
# Creates a new method named, for example,
# "stashed_defuse_bomb", which does the same
# thing as "defuse_bomb"
alias_method stashed_method, method_name
# Now we redefine "defuse_bomb" to do this:
define_method method_name do
begin
puts "==== #{Time.now} ===="
puts "About to execute '#{method_name}'..."
# Calls "stashed_defuse_bomb" to get the original version
send(stashed_method)
puts "Executed '#{method_name}' successfully."
rescue Exception => e
puts "Error while executing '#{method_name}': '#{e.message}'!"
ensure
puts "\n\n"
end
end
end
# Remove the lock
@adding_with_metaprogramming = false
end
end
end
</pre><br />
Metaprogramming! It sounds terrifying, doesn't it? Like, if programs can write programs, next thing you know, the Terminator is going to be ripping through your door! But it's actually <b>slightly</b> less terrifying than that.<br />
<br />
Anyway, I got the idea for the last post from reading one chapter in "Eloquent Ruby," and I got the idea for this post while lying in bed after reading another chapter. So: thumbs up, Eloquent Ruby! I hereby recommend you.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-45329496295381270712011-06-10T17:10:00.003-04:002011-06-10T17:14:08.144-04:00Using the Execute Around pattern in RubyAt my last job, I wrote a script that was run by a cron job overnight. The script was supposed to go through a MySQL table of scheduled database queries and, if they met some sanity checks, run them. This was so we could run slow queries like <span class="code">ALTER TABLE</span> during the wee hours of the morning, when customers weren't hitting our database. It may sound odd to execute MySQL queries that had been stored in MySQL, but it worked. (Just don't drop the stored queries table!)<br />
<br />
The first time my script ran, it failed, and I didn't know why. My boss wisely suggested that I add some logging: print messages to standard output. We were a PHP shop, so that meant <span class="code">echo</span> statements; all this output was captured in a log file by the cron job, which ran my script like this:<br />
<br />
<div class="code">query_runner.php > some_log_file.txt</div><br />
I went crazy with the logging, announcing up front how many queries I had to run, my progress ("Running query 4 of 7..."), how long each query took, whether it had errors, etc. At the end I printed a nice summary.<br />
<br />
All this made the log files very useful when something went wrong: I could tell which queries had and hadn't run, and generally why. I learned the value of logging, and that day, I became a man.<br />
<br />
No wait, I'm confusing my stories. I just learned the value of logging, that's all.<br />
<br />
<h3>Execute Around</h3><br />
Anywho, these days I'm coding Ruby, and I'm currently reading "Eloquent Ruby" by Russ Olsen. In one section, he shows a nice way of adding logging to any method you like, without cluttering up the internals of the method itself.<br />
<br />
The secret is the "execute around" pattern: you pass your method, as a block, to another method, which can do things before and after executing it. In this case, the logging method will log a message, execute the block, log another message, and even catch and log any exceptions.<br />
<br />
Here's an example:<br />
<br />
<pre class="brush: ruby">module Logging
def with_log(description, &block)
begin
puts "==== #{Time.now} ===="
puts "About to execute '#{description}'..."
block.call
puts "Executed '#{description}' successfully."
rescue Exception => e
puts "Error while executing '#{description}': '#{e.message}'!"
ensure
puts "\n\n"
end
end
end
class DangerBox
include Logging
def defuse_bomb
with_log('defuse_bomb') {puts "Snip the red wire..."}
end
def throw_error
with_log('throw_error') do
puts 'I feel an error coming on...'
raise 'Success: an error!'
end
end
end
</pre><br />
Here's what it looks like in action:<br />
<br />
<pre class="brush: ruby">d = DangerBox.new
d.defuse_bomb
# ==== Fri Jun 10 16:41:54 -0400 2011 ====
# About to execute 'defuse_bomb'...
# Snip the red wire...
# Executed 'defuse_bomb' successfully.
d.throw_error
# ==== Fri Jun 10 16:41:54 -0400 2011 ====
# About to execute 'throw_error'...
# I feel an error coming on...
# Error while executing 'throw_error': 'Success: an error!'!
</pre><br />
Lovely! All the logging and error handling is done in one place, and the methods that use it can be much cleaner.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-54614207327156788812011-06-09T17:19:00.001-04:002011-06-10T17:28:22.292-04:00Understanding Rack applicationsI'm starting to understand Ruby's Rack application architecture, and it's really cool. It's a simple concept, but very powerful, because of the ways it lets you stack up different components with different responsibilities. <br />
<br />
First, the basics. According to <a href="http://rack.rubyforge.org/doc/files/SPEC.html">the spec</a>:<br />
<br />
<blockquote>A Rack application is an Ruby object (not a class) that responds to call. It<br />
takes exactly one argument, the environment and returns an Array of exactly<br />
three values: the status, the headers, and the body.<br />
</blockquote><br />
So each component in a rack stack <b>is like a rung in a ladder</b>. An incoming request is stashed inside a hash called the environment, which includes info about the request itself and othert things (see the spec). That environment is passed down the ladder from one component to the next until it gets to the app at the end. The app receives the environment, generates a response, and returns it. Since the app was called by the final rack component, it hands it response back to that component. That component then returns the response to the component which called it, etc, effectively passing it back up the ladder, until it's returned to the user.<br />
<br />
Got that? <b>Requests go down the ladder, responses come back up</b>.<br />
<br />
Of course, in that description, each component is just passing the environment down and then passing the response back up. If that's all a component did, it would be useless. To be useful, it will have to actually something. Specifically, <b>a given component will do (at least) one of three things</b>:<br />
<br />
<ol><li><b>Screen the requests</b>. For example, if the component is for authorization, it verifies that the user has entered a valid login. If it's an invalid login, it will return a "failed login" response and never call the next component. If it's a valid login, it will pass the request to the next component - maybe a Rails app.</li>
<li><b>Modify the environment</b> before passing it on to the next component. Our authorization component might do this in addition to screening: if there's a valid login, it might set a user object in the environment before passing that to the next component. With this setup, the next component can assume that the user object will always be set whenever it gets a request.</li>
<li><b>Modify the response</b> after getting it back from the next component. A silly example would be to translate the response into Japanese.</li>
</ol><br />
Notice that each component needs to know what the <strong>next</strong> component in the chain is, so that it knows who to call. That's why whatever you give to Rack::Builder#use needs to have an <code>initialize</code> method that accepts the next component, in addition to a <code>call</code> method that accepts the env. <br />
<br />
The application at the end, however, doesn't need to call anybody else; it just needs to take the env as it then stands and generate a response. So whatever you pass to <code>Rack::Builder::run</code> doesn't need an <code>initialize</code> method; it just needs to respond to <code>call(env)</code>. Even a lambda can do that.<br />
<br />
Some pseudocode examples of call methods:<br />
<br />
<pre class="brush: ruby"># A component with this call method would be useless.
# Still, it's worth remembering that this:
def call(env)
@next_component.call(env)
end
# ... is Ruby shorthand for this:
def call(env)
result = @next_component.call(env)
return result
end
# Instead, we can do something useful on the way in:
def call(env)
valid_user = contains_valid_login?(env)
if valid_user
add_current_user_object_to(env)
# Next component can rely on having a current_user object
@next_component.call(env)
else
redirect_to_login_page_with_error
end
end
# Or we can do something useful on the way out:
def call(env)
result = @next_component.call(env)
result.translate_to_russian!
end
</pre>Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-53243372580594669192011-06-03T09:22:00.001-04:002011-06-03T09:46:56.244-04:00Ruby's Set classYesterday at work, we ran into an interesting problem. We're creating the new version of an application and discarding the old, ugly code. But we need to migrate some data: the old system has (let's say) widgets, and the new system has widgets, too. The old system uses 5 different databases (see how ugly?) with weird row schemas, but it does reliably have widget color, size, and shapes. The new system uses one database and has a nice row schema, but it also has widget color, size, and shapes.<br />
<br />
We need to know: which widgets are only in the old system? Which widgets are only in the new? Which are in both?<br />
<br />
<h3>Enter Sets</h3><br />
Asking these questions tripped a switch in my mind. "I know about this!", I thought. "This is a job for sets! And Ruby has a Set class."<br />
<br />
I'd never used them yet, but sets are made for this kind of thing. <a href="http://en.wikipedia.org/wiki/Set_(mathematics)">Sets</a> are often illustrated with <a href="http://en.wikipedia.org/wiki/Venn_diagram">Venn diagrams</a>: overlapping circles, where you ask "which things are only in the left circle? What's in the overlap?", etc.<br />
<br />
For instance:<br />
<br />
<img src="http://nathanlong.webfactional.com/blogimages/oneoffs/sets_venn_diagram.png"><br />
<br />
A set is a list of items where no item is repeated. If you have more than one set, you can compare them and answer the kinds of questions we've been asking. Here's a demo I just threw together:<br />
<br />
<pre class="brush: ruby">require 'set'
def sets_demo
# Sets ignore duplicate values
game_words = Set.new(['duck','duck','duck','goose'])
puts "Unique game words : #{game_words}\n\n"
#=> Unique game words : goose, duck
# Here are two sets with one thing in common
fast = Set.new(['bullet', 'cheetah'])
round = Set.new(['bullet', 'beach ball'])
# All the ways we can compare them
puts "Round : #{round}"
#=> Round : bullet, beach ball
puts "Fast : #{fast}"
#=> Fast : cheetah, bullet
puts ''
puts "Round and Fast (&) : #{(fast & round)}"
#=> Round and Fast (&) : bullet
#
puts "Round but not Fast (-) : #{(round - fast)}"
#=> Round but not Fast (-) : beach ball
puts "Fast but not Round (-) : #{(fast - round)}"
#=> Fast but not Round (-) : cheetah
puts "Round OR Fast (|) : #{(round | fast)}"
#=> Round OR Fast (|) : cheetah, bullet, beach ball
puts "Round OR Fast, but NOT both (XOR) : #{((round | fast) - (fast & round))}"
#=> Round OR Fast, but NOT both (XOR) : cheetah, beach ball
end
# Formatting the way the sets print
class Set
def to_s
to_a.join(', ')
end
end
sets_demo
</pre><br />
Got it?<br />
<br />
In my examples, the items in the sets were strings, but they could be anything. In our case at work, we used hashes: a widget was represented by a hash containing its color, shape and size. So, we just had to:<br />
<br />
<ol><li>Connect to each of the databases in the old system, getting all the widgets, creating a hash for each one, and dropping each into an old_system_widgets set (which automatically ignores duplicates)</li>
<li>Connect to the new system's database and make a similar set of its widgets</li>
<li>Do the kinds of set operations illustrated above</li>
</ol><br />
Voila! Now we knew which widgets were new and which ones still needed to be migrated to the new system.<br />
<br />
In conclusion: sets are swell!<br />
<br />
Hmmm. That's a pretty weak ending.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-85462523033246862062011-05-04T06:37:00.002-04:002011-05-05T09:03:16.615-04:00Ruby's String#unpack and Array#pack and UnicodeAt work, I'm exporting some data for use by a system that doesn't understand Unicode. I ran across <a href="http://www.jroller.com/obie/entry/fix_that_tranny_add_to">an Obie Fernandez post</a> explaining his approach.<br />
<br />
One method he showed was this:<br />
<br />
<pre class="brush: ruby">def to_ascii_iconv
converter = Iconv.new('ASCII//IGNORE//TRANSLIT', 'UTF-8')
converter.iconv(self).unpack('U*').select{ |cp| cp < 127 }.pack('U*')
end
</pre>
Essentially, this means that Iconv will convert the characters that it can from Unicode to ASCII. For example, <span class="code">รก</span> will be converted to <span class="code">'a</span>. (I don't know where to find a comprehensive list of these transliterations.)
After the transliteration step, any remaining non-ASCII characters - for example, Japanese characters, which have no ASCII equivalent - are discarded using the pack and unpack methods.
To see how this works, try this in irb:
<pre class="brush: ruby">"abcdefg".unpack('U*')
</pre>Your string is converted into an array of numbers representing their Unicode values, as we requested by using "U*". (I'm still a bit fuzzy on exactly how Unicode values work, but let's keep rolling.)
Ruby's Array#pack method can do the opposite conversion: numbers to string values.
<pre class="brush: ruby">(1..127).to_a.pack('U*')
</pre>There you should see a string of all the legal ASCII values, which, apparently, are all in the 1 to 127 range.
Knowing this, it's easy to see how you can throw away non-ASCII values in a string:
<pre class="brush: ruby">some_string.unpack('U*').select{|character_value| character_value < 127}.pack('U*')
</pre>
And that's what Obie's code does after its initial conversions with Iconv.
Now, if you're curious, you might want to see what Unicode values some other numbers map to. No problem: just change the range value from our earlier example, and write the resulting values to a file (in my case, the Unicode characters don't show correctly in irb):
<pre class="brush: ruby">string = (1..300).to_a.pack('U*')
File.open('unicodes_hooray.txt','w'){|f| f.write(string)}
</pre><br />
Open that up in an editor that can display Unicode to see what you got.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-31228551787733774102011-03-31T12:11:00.001-04:002011-03-31T12:44:39.829-04:00Authorized_keys and SCPI have SSH access to a machine, and I want to add my public key to the `authorized_keys` file so I don't have to type a password to log in. A friend taught me a neat command for doing this:<br />
<br />
<div class="code">% cat ~/.ssh/id_rsa.pub | ssh user@server "cat >> .ssh/authorized_keys"<br />
</div><br />
Meaning:<br />
<br />
<ul><li>Output the contents of my public key</li>
<li>Send that as standard input to this ssh command</li>
<li>The ssh command runs `cat` on the server</li>
<li>Since `cat` on the server is called without arguments, it uses standard input - in this case, the contents of my public key file</li>
<li>My public key gets added to `authorized_keys`</li>
</ul><br />
Added my key to `authorized_keys` has another benefit, besides not having to type the password when I SSH in: since I'm automatically recognized as a user, <b>I can use tab completion for scp commmands</b>. (Note: I'm using the zsh shell; remote tab-completion doesn't work for me in bash.)<br />
<br />
In other words, if there's a file called "foo.txt" in my home directory on that server, and I want to copy it to my local machine, I can start type this:<br />
<br />
<div class="code">% scp user@server:fo <br />
</div><br />
... and hit tab, and it will auto-complete "foo.txt". <b>That makes using scp a lot less tedious</b>.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-39061468793993789122011-03-31T10:15:00.002-04:002011-03-31T13:20:39.067-04:00Using git cherry-pickOne Git command that I've found to be really useful lately is "git cherry-pick". It lets you take a single commit from one branch and apply it to another.<br />
<br />
For example, say you're working on your "add_blinking_lights" feature branch, and you find a bug in the "rotate_missile_chamber" method. You fix it and commit to the branch between a couple of other feature commits.<br />
<br />
It's going to be a while before you merge this feature back into your master branch, but you do want that bugfix to go live on the next deploy. So, on the feature branch, you do this:<br />
<br />
<div class="code">cyborg[add_blinking_lights]% git log -3<br />
e9dd159 Added tests for blink rate<br />
a8e358c Fixed bug in rotate_missile_chamber<br />
a7d3302 Added tests for lights turning on<br />
</div><br />
OK, so you know the hash for the bugfix. Now you do "git checkout master."<br />
<br />
On the master branch you run "git cherry-pick a8e358c". Here's what that command does:<br />
<br />
<ul><li>Look at that commit on the feature branch</li>
<li>See what changes it introduced</li>
<li>Create a new commit on the master branch that introduces the same changes</li>
</ul><br />
Notice that it creates <b>a new commit</b> on the master branch. If, on master, you run "git log", you'll see <b>a different hash for the same commit message</b>. Why?<br />
<br />
This is because of how Git models what a commit is. A commit is a complete snapshot of the whole repository, and the hash for a given commit reflects the state of every file in the whole directory - it is a hash of all their hashes.<br />
<br />
So clearly, since master branch doesn't have all of the commits from the feature branch, a complete snapshot of it at the time the bugfix is applied will generate a different hash than a complete snapshot of the feature branch at the time the bugfix is applied there. Thus, different hashes.<br />
<br />
But when you do merge the feature branch into master, that won't matter; the hashes for the individual file where you made the bugfix will be the same, because their contents will be the same, so there will be nothing to update on master for that file.<br />
<br />
Of course, another way of approaching this problem would have been to switch to master before committing the bugfix in the first place, then merge master into the feature branch, or cherry-pick from master to the feature, or rebase the feature on master. But the method above is simple, and if you did already commit the bugfix on your branch, is the best way to get that commit into production.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-33112634744711536592011-03-10T20:35:00.000-05:002011-03-10T20:35:32.839-05:00Where cliche is a mandateA recruiting company that spams me has the slogan "Where Commitment is a Passion."<br />
<br />
What does that even <b>mean</b>?<br />
<br />
"We're always looking for stuff to commit to. I just committed to eating a whole jar of pickles! Jimmy over there committed to sing everything he says! Linda committed to paying the mortgage for the zoo! We're c-c-crazy for commitment!"Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-90310589894756408522011-03-07T15:30:00.003-05:002011-03-12T14:16:23.725-05:00Short Rubyzip tutorial<a href="http://rubyzip.sourceforge.net/">Rubyzip</a> is a handy tool for creating zip files with a Ruby script. But to me, the documentation doesn't make it immediately obvious how to use the thing. This quick tutorial should get you started.<br />
<br />
Say you're submitting some pictures of kitchen gadgets to Strange Cooks Weekly. You need to zip up your images before sending them, and <b>you're contractually obligated to use Ruby to do it</b>. Here's how.<br />
<br />
<pre class="brush: ruby">require 'rubygems'
require 'zip/zip'
puts "Zipping files!"
file_path = "Users/me/Pictures/kitchen_implements"
file_list = ['toast_mitten.png', 'gravy_jug.png', 'biscuit_juicer.png']
zipfile_name = "/Users/me/Desktop/someFile.zip"
Zip::ZipFile.open(zipfile_name, Zip::ZipFile::CREATE) do |zipfile|
file_list.each do |filename|
# Two arguments:
# - The name of the file as it will appear in the archive
# - The original file, including the path to find it
zipfile.add(filename, file_path + '/' + filename)
end
end
</pre><br />
Note that if the zip file already exists, this script will try to add stuff to it. If you try to add a file that's already in the archive, it will complain.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-17251078810216433132011-03-04T13:43:00.001-05:002011-03-12T14:17:15.017-05:00Using Ruby for SCP file transfersToday I needed to write a Ruby script to fetch a file using SCP. I thought it would be nice if it also displayed a "percent complete" counter as it went. This is what I came up with.<br />
<br />
<pre class="brush: ruby">require 'rubygems'
require 'net/scp'
puts "Fetching file"
# Establish the SSH session
ssh = Net::SSH.start("IP Address", "username on server", :password => "user's password on server", :port => 12345)
# Use that session to generate an SCP object
scp = ssh.scp
# Download the file and run the code block each time a new chuck of data is received
scp.download!("path/to/file/on/server/fileName", "/Users/me/Desktop/") do |ch, name, received, total|
# Calculate percentage complete and format as a two-digit percentage
percentage = format('%.2f', received.to_f / total.to_f * 100) + '%'
# Print on top of (replace) the same line in the terminal
# - Pad with spaces to make sure nothing remains from the previous output
# - Add a carriage return without a line feed so the line doesn't move down
print "Saving to #{name}: Received #{received} of #{total} bytes" + " (#{percentage}) \r"
# Print the output immediately - don't wait until the buffer fills up
STDOUT.flush
end
puts "Fetch complete!"
</pre>Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-7878002060319108572011-02-26T10:34:00.000-05:002011-02-26T10:34:55.889-05:00Best error message everA while back, I sent a text message and got this error back from AT&T.<br />
<a class="leftfloat" href="http://nathanlong.webfactional.com/blogimages/error_i_love_you.jpg"><img src="http://nathanlong.webfactional.com/blogimages/error_i_love_you.jpg"></a><br />
<br />
Wow... thanks. I... didn't know you cared?<br />
<br />
Apparently they just append the text of the message (this one was to my wife) to the end of the error, without saying "original message follows" or anything like that.<br />
<br />
In case anyone wants to try getting their own amusing messages, I believe this happened because I mistakenly sent to a land line.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-81363034047328419852011-02-23T17:05:00.001-05:002011-03-12T14:18:29.666-05:00Watching your log for a specific valueIf you use Unix, you probably know about piping one command's output through another. You may also know that <pre class="brush: bash">tail -f somefile</pre>will show you whatever gets added to the end of a file in real time.<br />
<br />
While working with Rails, I do <pre class="brush: bash">tail -f log/development.log</pre>all the time to watch how Rails processes my page requests. Today, I was looking for a specific error message to show up - a MissingTemplate error that a user had experienced**, and I was trying to reproduce it.<br />
<br />
Since I didn't care about anything that was being logged at that moment other than the error I was looking for, it occurred to me to do this:<br />
<br />
<pre class="brush: bash">tail -f log/development.log | grep 'MissingTemplate'
</pre><br />
As I poked around our site, I saw nothing. Until I encountered the error, and BAM - that one line popped into my terminal. Error case located!<br />
<br />
Just one more way that chaining simple tools together can be very powerful.<br />
<br />
*<i>*Actually, the user didn't see the MissingTemplate error; they got a friendly error message, as they should. But we got an email from Hoptoad with the actual error.</i>Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-55922561064844448592011-01-11T16:01:00.000-05:002011-01-11T16:01:51.025-05:00PHP: now running in the browser!Got this email from a recruiter just now:<br />
<br />
<blockquote>Hope you are doing well. We have an urgent requirement for<br />
<b>PHP Front End Developer</b> . Please go through the below mentioned job opportunity and let me know your availability for the same.</blockquote><br />
PHP Front End Developer!? Wow, those zany Facebook developers must be at it again. First they got PHP to compile, now they've got it running in the browser!Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-25210700683989035122011-01-06T12:32:00.000-05:002011-01-06T12:32:09.644-05:00IE: Still the biggest yak to shaveMy conclusion after a day and a half of fixing IE-only Javascript bugs.<br />
<br />
<img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMNpd38GDWPSerxYzMSsZb3jN7lvGuc5uUNxKISYrE68DvgFWCyQOqLJ_CbMA3pudvmThFrnCBS8GLJCyQ4lvU3tliMG9oF3L2O1QGbug3xdxy2Pg7jcuCa-X0B3Mqv2gDt-YIQdEago5g/s800/ie_yak.png"><br />
<br />
And this is IE8, mind you.<br />
<br />
Come on, Microsoft. Just use Webkit, already.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-82210053251195817222010-12-28T22:02:00.000-05:002010-12-28T22:02:01.850-05:00Introducing BindlerRails. Hobos. Bundler. Bindles.<br />
<br />
I awoke one day last week with this comic idea percolating in my brain. Too bad I'm lousy at drawing.<br />
<br />
<img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLkdUjKpnylpHpFj9vNlOCnUkKh2T_6XmuX02c0TRGmLjuetqPsrGKlvpCSlwuVGhev-tJMmvCgPSCpoEi6TK2KGCT356EDQvN0VV6Cj-VyMTZ-XJpELtt7bPtudGI4Q2Az92JxBaJRINk/s640/bindler-12-21-2010.png" />Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-26849367007990037882010-12-06T09:02:00.001-05:002011-03-12T14:20:25.200-05:00Chaining jQuery PseudoselectorsI just finished a very simple Javascript tweak to an app. Here was the requirement: "When the user opens our sign-in modal, we should focus the on the first input." Whatever code I wrote, I would put it a callback when the modal was finished opening.<br />
<br />
This was my first solution: target the id of the first input, and focus on it.<br />
<br />
<pre class="brush: js">$('input#user_email').focus();
</pre><br />
But I didn't like that, because it's not very flexible. What if we add another input to the beginning of the form? We'd have to change the javascript, too.<br />
<br />
After a little tinkering with jQuery's psuedoselectors, I came up with this:<br />
<br />
<pre class="brush: js">$('#sign-in-modal-form').find('input:not(:hidden):first').focus();
</pre><br />
Besides being more flexible to future changes, I think this is still pretty readable: select the first input that isn't hidden and focus on it.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-508963611403592392010-08-22T12:15:00.002-04:002011-03-12T14:21:09.404-05:00Seeing where Ruby methods come fromWhen it comes to reflection - asking an object or class to tell you about itself - Ruby makes a lot of sense. If you want to know what methods an object has, for example, you just ask it: <pre class="brush: ruby">someobject.methods</pre>.<br />
<br />
Any array has lots of handy, built-in methods, which it gets from the Kernel module (mixed in by every object), the Enumerable module, and Array itself. (For example, <span class="code">somearray.shuffle</span> returns a randomized version of the array.)<br />
<br />
To see what comes from where, we'll make an array, and ask it: "array, please get me an alphabetized list of your methods, go through that list, and print a statement saying where each one was defined." (Thanks to several respondents on Stackoverflow for <a href="http://stackoverflow.com/questions/3492679/ruby-determining-method-origins">showing me</a> this.)<br />
<br />
The resulting list presents all kinds of opportunities for exploration. (For example, <a href="http://ruby-doc.org/docs/ProgrammingRuby/html/taint.html">"tainted?" is very interesting</a>.)<br />
<br />
<pre class="brush: ruby">irb(main):001:0> a = Array.new
=> []
irb(main):002:0> a.methods.sort.collect {|m| puts "#{m} defined by #{a.method(m).owner}"}
& defined by Array
* defined by Array
+ defined by Array
- defined by Array
<< defined by Array
<=> defined by Array
== defined by Array
=== defined by Kernel
=~ defined by Kernel
[] defined by Array
[]= defined by Array
__id__ defined by Kernel
__send__ defined by Kernel
all? defined by Enumerable
any? defined by Enumerable
assoc defined by Array
at defined by Array
choice defined by Array
class defined by Kernel
clear defined by Array
clone defined by Kernel
collect defined by Array
collect! defined by Array
combination defined by Array
compact defined by Array
compact! defined by Array
concat defined by Array
count defined by Array
cycle defined by Array
delete defined by Array
delete_at defined by Array
delete_if defined by Array
detect defined by Enumerable
display defined by Kernel
drop defined by Array
drop_while defined by Array
dup defined by Kernel
each defined by Array
each_cons defined by Enumerable
each_index defined by Array
each_slice defined by Enumerable
each_with_index defined by Enumerable
empty? defined by Array
entries defined by Enumerable
enum_cons defined by Enumerable
enum_for defined by Kernel
enum_slice defined by Enumerable
enum_with_index defined by Enumerable
eql? defined by Array
equal? defined by Kernel
extend defined by Kernel
fetch defined by Array
fill defined by Array
find defined by Enumerable
find_all defined by Enumerable
find_index defined by Array
first defined by Array
flatten defined by Array
flatten! defined by Array
freeze defined by Kernel
frozen? defined by Array
grep defined by Enumerable
group_by defined by Enumerable
hash defined by Array
id defined by Kernel
include? defined by Array
index defined by Array
indexes defined by Array
indices defined by Array
inject defined by Enumerable
insert defined by Array
inspect defined by Array
instance_eval defined by Kernel
instance_exec defined by Kernel
instance_of? defined by Kernel
instance_variable_defined? defined by Kernel
instance_variable_get defined by Kernel
instance_variable_set defined by Kernel
instance_variables defined by Kernel
is_a? defined by Kernel
join defined by Array
kind_of? defined by Kernel
last defined by Array
length defined by Array
map defined by Array
map! defined by Array
max defined by Enumerable
max_by defined by Enumerable
member? defined by Enumerable
method defined by Kernel
methods defined by Kernel
min defined by Enumerable
min_by defined by Enumerable
minmax defined by Enumerable
minmax_by defined by Enumerable
nil? defined by Kernel
nitems defined by Array
none? defined by Enumerable
object_id defined by Kernel
one? defined by Enumerable
pack defined by Array
partition defined by Enumerable
permutation defined by Array
pop defined by Array
private_methods defined by Kernel
product defined by Array
protected_methods defined by Kernel
public_methods defined by Kernel
push defined by Array
rassoc defined by Array
reduce defined by Enumerable
reject defined by Array
reject! defined by Array
replace defined by Array
respond_to? defined by Kernel
reverse defined by Array
reverse! defined by Array
reverse_each defined by Array
rindex defined by Array
select defined by Array
send defined by Kernel
shift defined by Array
shuffle defined by Array
shuffle! defined by Array
singleton_methods defined by Kernel
size defined by Array
slice defined by Array
slice! defined by Array
sort defined by Array
sort! defined by Array
sort_by defined by Enumerable
taint defined by Kernel
tainted? defined by Kernel
take defined by Array
take_while defined by Array
tap defined by Kernel
to_a defined by Array
to_ary defined by Array
to_enum defined by Kernel
to_s defined by Array
transpose defined by Array
type defined by Kernel
uniq defined by Array
uniq! defined by Array
unshift defined by Array
untaint defined by Kernel
values_at defined by Array
zip defined by Array
| defined by Array
</pre>Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.comtag:blogger.com,1999:blog-8506302596139279537.post-5239119827004908932010-08-15T08:33:00.000-04:002010-08-15T08:33:40.472-04:00Visualizing our example setupSome people reading <a href="http://sleeplessgeek.blogspot.com/2010/01/setting-up-apache-php-mysql-phpmyadmin.html">my post on setting up Apache, PHP, MySQL, Filezilla Server and PHPMyAdmin</a> have seemed confused about how the whole setup works, or what the pieces are for. I put together this diagram a while back for a coworker and thought it might be helpful to readers here, too. Click the image for a larger version, suitable for zooming or printing out.<br />
<br />
<a class="leftfloat" href="http://nathanlong.webfactional.com/blogimages/http-ftp/Sleeplessgeek_-_FTP_and_HTTP_-_large.png"><img src="http://nathanlong.webfactional.com/blogimages/http-ftp/Sleeplessgeek_-_FTP_and_HTTP_-_thumb.png"></a><br />
<br />
The basic idea that I want to convey is that web traffic has two sides: a client and a server. All conversation between them passes over a network. When someone visits a web page, their client, the browser, says "hey, send me your web page, please!" And on the other end of the network, a server gets the request and responds with the web page.<br />
<br />
<h3>What's a server?</h3><br />
Before we go further, let's clear up some confusion about terms. Specifically, what is a server?<br />
<br />
<b>People use the term "server" to mean different things</b>. In some cases, we mean "the program that responds to requests." With that meaning in mind, we'd call Apache a web server, MySQL a database server, and Filezilla Server an FTP server. In other cases, we mean "the computer that the server program(s) run on." With that in mind, we might say "our server sits on a rack in the server room."<br />
<br />
In some setups, MySQL might be running on one server machine and Apache on another. In complex setups, both will be running on multiple machines. <br />
<br />
The tutorial was all about setting up the various pieces of server software. If you set them up on your own computer, then visit your site from that same computer, then you refer to the server you're connecting to as "localhost" - local in the sense that it's on the same machine. In that case, the top part (the server) and the bottom part (the client) are really the same computer, and the network request never travels outside the computer you're working on. But of course most web sites you visit are served from elsewhere, and the network request might travel around the world.<br />
<br />
Now, to look at the components we set up. Let's start by looking at the right side of the diagram, the HTTP side. <br />
<br />
<h2>HTTP</h2><br />
This is the conversation between a browser and the server program - in our case, Apache. HTTP is the protocol they use to communicate. This just means that each of them expects requests and responses to be formatted a certain way - with headers, a body, and various sub-components of each.<br />
<br />
When a user requests a URL, it's up to Apache to decide what to send back. In basic cases, if the user asks for "http://servername/foo.html", Apache will look for foo.html, read it, and send back its contents. If you ask for "bar.html" and no such file exists, it will tell you so. But you could have Apache do anything you like. For example, it could send back the same message no matter what URL was requested. Or it could map URLs that match certain patterns to certain responses, without there being a one-to-one relationship between requests and files on the server. (Ruby on Rails has a complex "routes" system that does this.)<br />
<br />
Now imagine that the browser has requested a page, and Apache sends it back. What exactly does it send back? Just the HTML (in an HTTP "envelope", of course). The browser then looks at the HTML and says, "oh, I see that this lists some images, and a CSS file, and a Javascript file." It's then up to the browser (and the user's settings) whether it will ask for those other resources or not.<br />
<br />
If the browser requests, say, an image on the page, the server sends it back, too. As far as the server is concerned, each request is totally separate. You can imagine the conversation like this:<br />
<br />
<b>Browser</b>: I'd like this URL, please.<br />
<b>Server</b>: Oh, hi, stranger! OK, here's some HTML.<br />
<b>Browser</b>: (Hmmm, there's an image listed.) Server, please send me the image at this URL.<br />
<b>Server</b>: Oh, hi, stranger! OK, here's that image.<br />
<br />
Notice that the server doesn't "remember" the browser's previous request. In fact, it might even be a different server responding to the second request. This shows that HTTP is "stateless" - meaning that each request is separate, and doesn't depend on any previous ones. (This gets more complicated when you have logins and cookies and sessions, etc, but at bottom, it's the way HTTP works.)<br />
<br />
In our example setup, when you ask for an HTML file, Apache just finds it and sends back the contents "raw" - without changing them. The same thing is true for CSS and Javascript files. But PHP is different.<br />
<br />
<h3>Adding PHP</h3><br />
In our setup, we configured Apache to treat requests for .php files differently. When it sees that type of request, it asks php.exe to process the request and tell it how to respond.<br />
<br />
PHP.exe will look at the file and execute any of the PHP instructions, like including headers from another file, running functions, pulling information from classes or objects, etc.<br />
<br />
If the script requires it to, PHP will talk with a database server (in our case, MySQL) to get some of the information it needs to complete the request. For example, maybe it has to ask MySQL for a list of products, then output image tags and product descriptions for a catalog. <br />
<br />
When it has finished running the script, PHP.exe gives the complete web page output to Apache, which sends it back to the browser, which can then ask for the images if it pleases.<br />
<br />
One more thing to notice: <b>as far as the browser is concerned, PHP and MySQL don't exist</b>. It asks for a URL, and it gets back HTML (or an image, or other resource). It doesn't know or care how that gets put together. When it asks for "http://somesite.com/foo.php", it doesn't "know" that "foo.php" is a PHP script (and in fact, depending on how the server is configured, it might not be). The browser could ask for "foo.jpg" and get back HTML and it wouldn't be surprised, as long as the HTTP headers in the server's response said "what I'm sending you back is HTML."<br />
<br />
In our setup, "foo.php" gets run when the browser requests a URL and the URL path is the file path to that script. But this is just a convenience for us, which makes it easy to see how the process works. We could make it more complicated, and as long as the browser could make a request and get a response it understood, it wouldn't know or care.<br />
<br />
<h2>FTP</h2><br />
So where does Filezilla Server come in? If your server is on localhost, it may not. The basic purpose for Filezilla is to let you move files between the client and server computers. If they're the same computer, you can just copy the files there directly. But if not, you'll have to do it over the network, which is what FTP (File Transfer Protocol) is for.<br />
<br />
Looking at the left side of the diagram, you'll see that there's another client-server relationship. This time, Filezilla Server is the server, and an FTP client (which could be Filezilla Client) is the client. If Filezilla Server is configured to use the same folder of files as Apache, you'll be able to change the content of your web site using this FTP conversation. To add or update a script, you'll upload it via FTP. The next time Apache gets a request for it, it will find it, get PHP to process it and send the results to the browser.Nathan Longhttp://www.blogger.com/profile/02909682437213600859noreply@blogger.com