Objectives: The authors characterized the demographic, socioeconomic, and psychosocial work factors associated with performance of unskilled manual work, and then identified the modifiable psychosocial work factors that affected the psychological well-being of these workers. Material and Methods: This study analyzed data from the fifth Korean Working Condition Survey conducted in 2017. The study subjects were 37 081 Korean employees. The occupational classes investigated were: managers, professionals, and clerks; service and sales workers; and skilled or unskilled manual workers. Results: Unskilled manual workers were more likely to be elderly and less educated, to have low income, to work fewer hours weekly, to have a shorter work duration, to perform temporary or daily jobs, and to report poor subjective health and well-being. Unskilled manual workers were also more likely to experience psychosocial hazards, such adverse social behaviors, a lack of job satisfaction, a lack of support from managers, and a poor social climate. However, with statistical adjustment for demographic, socioeconomic, and psychosocial work factors, unskilled manual work was no longer associated with poor psychological well-being, but psychosocial work factors were associated with poor psychological well-being. Conclusions: The poor psychological well-being of unskilled manual workers cannot be explained by the intrinsic nature of this type of work. Instead, the poor psychological well-being of these workers is associated with unfavorable psychosocial work factors, such as a poor employment status, a lack of job satisfaction, a lack of support from managers, and a bad social climate. These results thus suggest that the modification of psychosocial work factors may improve the psychological well-being of unskilled manual workers. Int J Occup Med Environ Health. 2021;34(6):789–804