drop the prop off, undo the screw. the drum should slide off, no levering needed. if it doesnt, lever one of the rubber caps out (if they are still there) and look inside for the adjuster, lever this round a few times to bring the shoes inward, and the drum will come off.
i dont know if owt will drop off, coz i dont know whats up with your handbrake. it shouldnt do if all well in there.