This paper proposes an adaptive error diffusion algorithm for text enhancement followed by an efficient text segmentation that uses the maximum gradient difference (MGD). The gradients are calculated along with scan lines, then the MGD values are filled within a local window to merge text segments. If the value is above a threshold, the pixel is considered as potential text. Isolated segments are then eliminated in a non-text region filtering process. After the text segmentation, a conventional error diffusion method is applied to the background, while edge enhancement error diffusion is used for the text. Since it is inevitable that visually objectionable artifacts are generated when using two different halftoning algorithms, gradual dilation is proposed to minimize the boundary artifacts in the segmented text blocks before halftoning. Sharpening based on the gradually dilated text region (GDTR) then prevents the printing of successive dots around the text region boundaries. The method is extended to halftone color images to sharpen the text regions. The proposed adaptive error diffusion algorithm involves color halftoning that controls the amount of edge enhancement using a general error filter. However, edge enhancement unfortunately produces color distortion, as edge enhancement and color difference are trade-offs. The multiplicative edge enhancement parameters are selected based on the amount of edge sharpening and color difference. Plus, an additional error factor is introduced to reduce the dot elimination artifact generated by the edge enhancement error diffusion. In experiments, the text of a scanned image was sharper when using the proposed algorithm than with conventional error diffusion without changing the background.