Graphite and Diamond are different because they have different structures. Both have Giant Covalent Structures, resulting in very high melting temperatures. However each carbon atom in Diamond has 4 covalent bonds with other Carbons, making it extremely strong and hard. On the other hand, each carbon in graphite is bonded to three carbons, and therefore graphite is formed in layers.
You may think Graphite is soft because it is used in pencils but the layers themselves are actually very strong, however because each carbon only has three bonds, the layers have delocalised 'free' electrons between them. These electrons act as lubricant between layers, allowing them to slide over one another, making graphite appear soft. The free electrons also mean that Graphite conducts electricity. Diamond does not have these free electrons so does not conduct electricity.