[Lowerbounds, Upperbounds]

Algorithms are everywhere.

http://games.slashdot.org/article.pl?sid=05/04/24/1259241

Unlike normal Tetris(R), however, Bastet does not choose your next brick at random. Instead, Bastet uses a special algorithm designed to choose the worst brick possible.

Surely this situation will sound too familiar among the algorithm designers. :P

I also checked my Worst-Case Scenario Survival Handbook. No, it doesn’t tell me how to survive Bastet. My high score so far is 5 lines, after trying about 10 games. At last, there is a game that I will happily accept my low scores.

P.S. Although not directly related, this reminds me of Tetris is Hard, Even to Approximate by Demaine, Hohenberger and Liben-Nowell.

Michael Nelson has a nice introductory series written on expander graphs. I particularly like the fact that Michael has taken the time to “boil down” the proofs. Thank you!

http://www.qinfo.org/people/nielsen/blog/?p=224

Dear Expanders, are you really all algebraic?

In Part 1, I have shown how you can increment a “binary” counter in `O(1)` time if you use a redundant number system. Now I am going to show you how you can apply this simple abstraction to deamortize Binomial Heaps (here is the Wikipedia page for Binomial Heaps if you want a refresher).

Where does the amortization come in for Binomial Heaps? Well, all the time bounds are indeed stated as worst-case bounds. But, as is commonly taught in many places, you can have a more precise time bound for Insert if you do an amortized analysis. The argument is particularly simple if we use a binary counter abstraction.

We will encode the configuration of the binomial trees using a binary counter `d_{log n}…d_1d_0`. We set `d_i` to be 1 iff a binomial tree of rank `i` is present. Inserting into a binomial heap is now precisely incrementing this counter. Linking two binomial tree of rank `i` to form a binomial tree of rank `i+1` is exactly converting a 2 in `d_i` to a 1 in `d_{i+1}`. Guess there must be a reason why it’s called a binomial heap. :)

The amortized analysis can be done using the credit invariant method. We argue that over a series of `n` insertions, the digits only changed at most `2n` times. For each rank (digit), we open a bank account. Each insertion pays two dollars: one spent immediately for the actual insertion and one deposited at the account of the first 0 that turned into a 1—observe that precisely one 0 will turn into a 1 in every increment as the propagation will stop immediately afterwards. The dollar deposited in the account of this 1 will be used to turn it back into a 0 during a future insertion.

Check that we will have enough money to pay for all digit changes during any insertion. As we never go bankrupt, we conclude that 2 dollars per insertion is enough. In other words, insertion actually takes “only” amortized `O(1)` time.

Equipped with our technology from Part 1, now we can implement insertion in worst-case `O(1)` time simply by using a redundant counter. Indeed we will allow two binomial trees of rank `i` to co-exist for a while, until the counter needs to link the two when fixing its regularity. Simple enough.

For those of you who want to know how redundant counters were invented in the first place, see

  • Michael J. Clancy and Donald E. Knuth. A Programming and Problem-Solving Seminar. Technical Report STAN-CS-77-606, Computer Science Department, Stanford University, April 1977.

Note that over the years many extensions to redundant counters have been developed, and in fact the terminologies used in this series are “modern”. We will talk more about those in the future.

In the positional number system of base `b`, the representation `d_{n}d_{n-1}…d_{1}d_{0}` denotes the number `\sum_{i = 0}^{n} d_i b^i`. For example, the binary (`b = 2`) number `(1001)_2` is 8 + 1 = 9. Notice that when `d_i \in {0, …, b – 1}`, there is only one way to denote any number. This is what we are all used to.

In a redundant number system, we allow `d_i` to be chosen from a larger set of digits. For this series, we will stick to `{0, …, b – 1, b}`. To make things even more concrete, we further set `b` to 2, i.e., we are considering the redundant binary numbers with the sets of digits {0, 1, 2}.

The decimal number 9 can still be written as `1001` in binary (I am dropping the subscripts and will display binary numbers in math mode). But since we are using the redundant binary system, we also have other choices. In particular, `201` (2 * 4 + 1 = 9) and `121` (4 + 2*2 + 1 = 9) are also valid representations.

Let me introduce three terms before I continue.

  • We say that a redundant binary number is regular if the 2s and 0s alternate. So `2012` is regular, but `2120` is not.
  • When the last non-1 digit of a redundant binary number is 0, we say that it’s 0-exposed. For example, `2001` is 0-exposed. (But note that it is not regular.)
  • When the last non-1 digit is 2, we say that it is 2-exposed. For example, `2102` is 2-exposed.

The regularity constraint is very special. First, convince yourself that even when restricting to regular representations, there are still redundancy in the representation of a number. For example, both `1202` and `1210` denote 18 and both of which are regular since the 2s and the 0s alternate.

Also note that if a number is either 2-exposed or 0-exposed.

The only operation that we will consider in this post is:

  • add 1 to the least significant digit of a number (i.e., the rightmost)

We will denote this operation with an intuitive expression. For example, consider `12(1+1)`. That would be `122`. Easy, huh?

Let’s repeat the operation one more time. Hmm, `12(2+1)` would be `123`, but that wouldn’t be too good since 3 is not an allowed digit. Well, we need carry. `123` = `131` = `211`. OK, problem solved.

At this point, make sure you know how to do this add-one-to-the-last-digit operation.

For those of us that own the whole series of the Worst-Case Scenario Survival Handbook :) , you will have noticed that this operation can take time linear to the number of digits. Consider the input `22222222222222222(2+1)`. Oh no!

This is where the regularity constraint comes in. Suppose the number you start with is regular. How long can add-1 take? Consider the last digit of the number.

  • If it’s a 0 or 1, then you can just bump it up to 1 or 2 respectively.
  • If it’s a 2, then we need to carry 1 to the next digit (to the left). Ah! But since the number is regular, that digit cannot be a 2. The previous case now applies.

So in at most two steps, we will be done. That’s like `O(1)` time, right?

But, we may not be left with a regular number any more. For example, `12(1+1)` is `122` and that’s not regular. The key observation is that we will end up with an irregular number if we start with a 2-exposed regular number. So we never want to increment a 2-exposed number.

The claim is that you can convert a 2-exposed number into a 0-exposed equivalent in `O(1)` time if you know where the rightmost 2 is. Let the number be written as `xy2z`, where `z` is the sequence of digits to the right of the rightmost 2, `y` is the digit immediate to the left of it, and `x` is the sequence to the left of `y`. Due to the regularity constraint, note that `z` must be all 1s if it is not null, and `y` only has two possibilities:

  • `x02z` = `x10z`
    `x` must be 2-exposed, so `x10z` is regular.
  • `x12z` = `x20z`
    `x` must be 0-exposed, so `x20z` is regular.

And both of these are 0-exposed and we know adding 1 to them takes `O(1)` time.

The remaining issue is how we know where the rightmost 2 is. This is easy. We maintain a stack of the locations of the 2s, with the rightmost on the top of the stack. Verify that the above two fixing operations both operate at the top of the stack.

Now do you understand how to add 1 in `O(1)` time?

Indeed, if the number is 2-exposed, fix it to become 0-exposed first. And we know that adding 1 to a 0-exposed number is easy.

But why am I telling you about redundant number systems? That would be the subject of Part 2.

Las Vegas, NV, USA
July 17-20, 2005

P.S. I made the mistake of booking my flights before actually reading the conference program. Thinking that the conference starts on the 17th, oh well, I booked my flight so that I will arrive in the afternoon of the 16th. No, I really did not plan to spend time in the casinos. :P

After reading a comment at Lance Fortnow’s blog, I suppose I should write more about Computer Science. :P To do that, first I followed the steps of our friendly alumus Andrej Bauer and installed ASCIIMathML.

The actual syntax of ASCIIMathML is not exactly LaTeX as the author Peter Jispen wants a syntax that is

  1. close to standard mathematical notation
  2. easy to read
  3. easy to type

Oh well, so much for the dollar signs. (But let me note that LaTeX syntax is supported to some degree.)

If you are using IE, then you need to install MathPlayer. If you are using Firefox, you want to visit the MathML page at Mozilla.org. In particular, you will need to install the fonts from their Fonts page.

Here is an example copied from the ASCIIMathML page for testing.

Solving the quadratic equation.
Suppose `ax^2+bx+c=0` and `a!=0`. We first divide by `a` to get `x^2+b/ax+c/a=0`.

Then we complete the square and obtain `x^2+b/ax+(b/(2a))^2-(b/(2a))^2+c/a=0`.
The first three terms factor to give `(x+b/(2a))^2=(b^2)/(4a^2)-c/a`.
Now we take square roots on both sides and get `x+b/(2a)=+-sqrt((b^2)/(4a^2)-c/a)`.

Finally we move the `b/(2a)` to the right and simplify to get
the two solutions: `x_(1,2)=(-b+-sqrt(b^2-4ac))/(2a)`

Now we are one step closer to make this blog more relevant to its title. :P

Since STOC 2005, there has been a lot of discussion about theory funding and a task force has been set up by SIGACT to work on advocacy issues.

To start, you are referred to the post by Suresh Venkatasubramanian and a slightly later post by Lance Fortnow. Make sure you read the comment by Michael Mitzenmacher for a clarification on the purpose of the task force, directly from a task force member.

Let the message spread.

P.S. Jeff Erickson has an interesting post about basic research funding too.

The list has been posted. And before I know it, Anupam has already updated his page.

http://www.cs.cornell.edu/Research/focs05/list.htm

From arXiv’s cs daily comes a theory paper that can be used to deal with matters of life and death. How could theory be any more practical than *that*? :P

The intuition here is that it is saying: “Regarding membership in L, if you put a gun to my head and forced me to bet on one of x or y as belonging to L, my money would be on f(x,y).”

Full information:
Read the rest of this entry »

Do you know that Perl “regular expressions” can accept NP-complete languages?

http://perl.plover.com/NPC/

Happy backtracking!

Don’t let Keanu Reeves steal the focus!

This is an excellent tutorial on Singular Value Decomposition (SVD) by Todd Will from UW-La Crosse. I highly recommend it for everybody who deals with Linear Algebra.

http://www.uwlax.edu/faculty/will/svd/index.html

Among other excellent insights, once you read the page on Perpframes, Aligner and Hangers (the 3rd page in that site), you will never see a matrix the same way again. It’s like the, erh, blue pill. (Or is it the red one? :P )

From arXiv cs daily comes a very interesting paper titled Online Medians via Online Bribery by Marek Chrobak, Claire Kenyon, John Noga and Neal E. Young. Here is part of the abstract:

Our proofs reduce online medians to the following online bribery problem: faced with some unknown threshold T>0, an algorithm must submit “bids” b>0 until it submits a bid as large as T. The algorithm pays the sum of its bids. We describe optimally competitive algorithms for online bribery.

I suppose this can be a useful skill when I travel to some parts of the world later this summer? :P

Full information:
Read the rest of this entry »

Check this out…. a new 10 page (only!) proof to the PCP theorem due to Irit Dinur:
http://eccc.uni-trier.de/eccc-reports/2005/TR05-046/index.html

Finally I can hope to try to understand the theorem! And supposedly it uses a purely combinatorial amplification lemma.