Server object extension coding practices
To create a map or image service server object extension (SOE), you typically write code that implements some required interfaces and performs your business logic. If you're working in Java, you can get started creating an SOE using an Eclipse plug-in wizard. If you're working in .NET, you'll start in Visual Studio with a template project. The wizard and templates ensure that your SOE implements the required interfaces and can respond to REST or SOAP web service calls using classes in SOESupport.
You can find detailed instructions for coding SOEs in the ArcObjects SDK:
It's also possible to create an SOE with C++ on Windows, and a sample is included in the .NET ArcObjects SDK for C++.
Working with map services
SOEs are only supported for extending map and image services. Because map and image services use a service definition file and do not access a map document (MXD) directly, there are certain classes you should avoid and others you should favor when writing SOEs.
Avoid using ArcObjects from the Carto library that are specifically designed for use with MXDs. This includes IMap, ILayer, and items related to data frames and page layouts. Instead, use ArcObjects designed for working with map services, such as MapServer, MapLayerInfos, and MapDescription. Use the interface IMapServerDataAccess to access the datasets underlying each layer in your map.
Libraries not directly related to the map document, such as ESRI.ArcGIS.Geometry and ESRI.ArcGIS.Geodatabase for .NET or com.esri.arcgis.geometry and com.esri.arcgis.geodatabase for Java, are always permissible to use in SOEs.
Creating an .soe file
Starting at ArcGIS 10.1 for Server, SOEs are encapsulated in an .soe file. If you're using .NET, the .soe file is created when you build your project from the SOE template. If you're using Java, you'll create the .soe file using a wizard built into Eclipse. The .soe file contains all the information necessary to register your SOE with ArcGIS Server.