The quantitative credit evaluation of enterprises is a cornerstone of establishing a more efficient market economy supervision system in China. However, the credit evaluation of medium, small, and micro enterprises has been challenging due to the lack of public business and financial data. Based on the observation that there are a large number of open and unstructured business registration data of medium, small, and micro enterprises on the Internet, this paper proposes to mine unconventional and unstructured data such as enterprise names, investment relationships between enterprises, and business scope to improve the credit evaluation results. Specifically, this paper proposes two data representation methods. The first uses a Gated Recurrent Neural Network to extract sequential information from the business registration text and transform the text into numerical data.The second uses a Graph Attention Network to encode the graph structure formed from the investment relationships into a numerical space. As a result, the heterogeneous information can be easily fused by merging the numerical vectors. Since the interpretability of credit evaluation models is crucial in financial applications, this paper further proposes interpretable solutions for the textminingbased credit evaluation model and for identifying the credit risk transmission path. The experimental results based on 68 504 enterprises revealed that both enterprise names and investment relationships contain credit information that cannot be identified in traditional numerical data. The results showed that business registration information can be used as a useful supplement to the current enterprise credit evaluation system, which is valuable in dealing with the scarcity of credit information for medium, small, and micro enterprises.