The reason for this is that the .join() method is a merge on the index, so if you want to compute a rather complex variable and put it back on the table, you either use merge and keep track of the key yourself, or simply use join.