We describe the Carnegie-Spitzer-IMACS (CSI) Survey, a wide-field, near-IR selected spectrophotometric redshift survey with the Inamori Magellan Areal Camera and Spectrograph (IMACS) on Magellan-Baade. By defining a flux-limited sample of galaxies in Spitzer \emph{IRAC} $3.6\mu$m imaging of SWIRE fields, the CSI Survey efficiently traces the stellar mass of average galaxies to $z\sim 1.5$. This first paper provides an overview of the survey selection, observations, processing of the photometry and spectrophotometry. We also describe the processing of the data: new methods of fitting synthetic templates of spectral energy distributions are used to derive redshifts, stellar masses, emission line luminosities, and coarse information on recent star-formation. Our unique methodology for analyzing low-dispersion spectra taken with multilayer prisms in \emph{IMACS}, combined with panchromatic photometry from the ultraviolet to the IR, has yielded high quality redshifts for 43,347 galaxies in our first 5.3 degs$^2$ of the SWIRE XMM-LSS field. We use three different approaches to estimate our redshift errors and find robust agreement. Over the full range of $3.6\mu$m fluxes of our selection, we find typical redshift uncertainties of $\sigma_z/(1+z) \la 0.015$. In comparisons with previously published spectroscopic redshifts we find scatters of $\sigma_z/(1+z) = 0.011$ for galaxies at $0.7\le z\le 0.9$, and $\sigma_z/(1+z) = 0.014$ for galaxies at $0.9\le z\le 1.2$. For galaxies brighter and fainter than $i=23$ mag, we find $\sigma_z/(1+z) = 0.008$ and $\sigma_z/(1+z) = 0.022$, respectively. Notably, our low-dispersion spectroscopy and analysis yields comparable redshift uncertainties and success rates for both red and blue galaxies, largely eliminating color-based systematics that can seriously bias observed dependencies of galaxy evolution on environment.