This paper is concerned with the algebraic structure of groupoids and crossed modules of groupoids. We describe the group structure of the automorphism group of a finite connected groupoid C as a quotient of a semidirect product. We pay particular attention to the conjugation automorphisms of C, and use these to define a new notion of groupoid action. We then show that the automorphism group of a crossed module of groupoids C, in the case when the range groupoid is connected and the source group totally disconnected, may be determined from that of the crossed module of groups Cu formed by restricting to a single object u. Finally, we show that the group of homotopies of C may be determined once the group of regular derivations of Cu is known.