Eureeka's Castle was a fantastic show for kids. It composed of muppet type characters that have many great musical moments combined with learning techniques for preschoolers. If you remember "Fraggle Rock" and "Muppet Babies" then you'll know exactly what this show was like. The writing was very strong and the performances were top-grade.
The Halloween episode, also called It Came From Beneath the Bed or Nightmare on Magellan Street, wasn't too scary for the little ones, but it was terror-laden without losing any of that colorful delight we came to expect from an Eureeka's Castle episode. I can't seem to find this episode on DVD or VHS, but you can watch it online below for free.