Eostre, the Goddess of Springtime, was bringing Spring to the land when she happened upon a young girl who was weeping in the forest. The girl had found a bird who had frozen to death in the snow. Taking pity on her, Eostre went to the girl to bring the bird back to life but Eostre could not save the bird's wings so she turned the bird into a rabbit instead.
Having once been a bird, the rabbit retained its ability to lay eggs and every Spring thereon the rabbit would lay eggs in all the bright colors of Spring as a thank you to the goddess Eostre for saving its life.
And that is the story of the Easter Bunny and the tradition of why we decorate eggs in bright colors. Eostre was the Goddess of Springtime and fertility hence the further symbolism of rabbits (procreation), eggs (fertility) and chocolate (aphrodisiac).