This is an end semester project that I have done with my groupmates in college. In this study, we proposed a model to predict a startup company’s future condition using a deep multilayer perceptron (MLP) and decision tree. I mainly played the role for data quantification and literature review of this study. In our study, we built a prediction analysis model for startup companies. Furthermore, we identified what the key factors are and what influences will it have on the results. Our main hypothesis is that money, people and active days are the key factors. We built the prediction model from CrunchBase, which is the largest public database with relevant profiles about companies.
Data Quantification
For the data obtained, there are 4 groups of data that are quantified: country state, employee range, roles and countries. Regarding the states group (which only exists in USA and Canada), we think that different states have different impacts on the survival rate of a startup. A lookup table for scoring of states is defined [1][2][3]. The original data for employee quantities are a range between two numbers (e.g. 101-250). These are transformed into the average of the upper and lower bound. Employees of 10000+ are defined to be 15000. For the roles group, the data “company” is arbitrarily defined as 0.1 and “company, investor” is defined as 0.9. This is because we regard a company as wealthier and influential when it also plays a role of an investor, compared with only being a company. The other roles are transformed to 0.5. The impact of country on startup environment is also studied [4] and scores from 0.329 to 0.947 are given to countries that have above 300 startup companies in record.
Table 1. Company status and scores of 23 countries.
| Country | Closed | Operating | Acquired | IPO | Score |
|---|---|---|---|---|---|
| BRA | 4.7% | 90.1% | 4.5% | 0.6% | 0.329 |
| ESP | 5.3% | 86.8% | 7.3% | 0.7% | 0.333 |
| FIN | 3.6% | 82.9% | 12.6% | 0.9% | 0.340 |
| ITA | 2.9% | 89.5% | 6.6% | 1.0% | 0.345 |
| RUS | 6.0% | 87.5% | 4.7% | 1.8% | 0.379 |
| KOR | 4.9% | 89.0% | 3.5% | 2.6% | 0.411 |
| BEL | 4.3% | 80.6% | 12.3% | 2.8% | 0.419 |
| NLD | 3.6% | 83.5% | 10.1% | 2.8% | 0.421 |
| GBR | 5.2% | 81.4% | 10.5% | 2.9% | 0.423 |
| DEU | 5.5% | 78.7% | 12.9% | 2.9% | 0.423 |
| IRL | 5.8% | 78.9% | 12.0% | 3.3% | 0.443 |
| DNK | 4.4% | 79.6% | 12.1% | 3.9% | 0.466 |
| CHE | 2.7% | 83.3% | 10.0% | 4.0% | 0.471 |
| SWE | 3.9% | 82.3% | 9.8% | 4.0% | 0.473 |
| USA | 8.1% | 69.3% | 18.5% | 4.2% | 0.478 |
| IND | 3.3% | 85.5% | 6.8% | 4.4% | 0.487 |
| SGP | 3.2% | 85.5% | 6.5% | 4.8% | 0.507 |
| FRA | 4.0% | 78.4% | 12.6% | 5.1% | 0.516 |
| ISR | 5.4% | 74.7% | 13.7% | 6.2% | 0.563 |
| JPN | 3.4% | 82.2% | 5.4% | 9.0% | 0.684 |
| CAN | 5.0% | 70.6% | 13.3% | 11.1% | 0.769 |
| CHN | 3.3% | 79.2% | 4.0% | 13.5% | 0.874 |
| AUS | 3.3% | 77.1% | 4.4% | 15.3% | 0.947 |
Neural Network Implementation
The ANN runs on a Windows 10 OS (i7-8700k CPU) with DDR4 2666MHz 16G RAM and Nvidia GTX 1070 Ti GPU. Keras is used to construct the network, and three networks of multilayer perceptron (MLP) with different number of layers are designed for comparison. There are 4 outputs of the network, indicating the probabilities of the final status which are: (1) Closed, (2) Operating, (3) Acquired and (4) IPO.
Figure 1. MLP Structure of the 3 neural networks.
The three networks have respectively 2, 4 and 6 hidden layers, with each network all starting with a 1024-neuron hidden layer and ending with a 32-neuron hidden layer (Fig. 1). Table 2 shows the training results.
Table 2. Mean square error (MSE) and categorical cross-entropy loss for the 3 networks.
| Loss | Activation | Network 1 | Network 2 | Network 3 |
|---|---|---|---|---|
| MSE | ReLU | 0.695 | 0.694 | 0.689 |
| MSE | sigmoid | 0.690 | 0.690 | 0.691 |
| MSE | linear | 0.692 | 0.689 | 0.692 |
| MSE | tanh | 0.688 | 0.693 | 0.691 |
| Cross-entropy loss | ReLU | 0.695 | 0.691 | 0.690 |
| Cross-entropy loss | sigmoid | 0.692 | 0.691 | 0.691 |
| Cross-entropy loss | linear | 0.695 | 0.692 | 0.690 |
| Cross-entropy loss | tanh | 0.691 | 0.692 | 0.691 |
The results show that almost all results are close to 70% with few difference. In general, ReLU activation has a better result than sigmoid activation, while linear activation may outperform ReLU when the network is deep enough.
The neural network behaves like a black box. It is quite difficult to conclude significant insights just by looking at the trained parameters. However, regarding the importance for business, we used a decision tree to help us understand which factor is the most important and how important they are respectively.
Decision Tree Training
We tried three different depths of decision tree: 4 (Fig. 2), 6 (Fig. 3), and 10. We set the gain ratio to be the criterion, and the confidence to 0.1.
Figure 2. Decision tree of depth = 4.
Figure 3. Decision tree of depth = 6.
The results show that if a company is large enough to exceed 500 people, the close rates are low. In most cases, large companies are acquired by mergers and acquisitions. In addition to the number of employees, funding total amount also has a significant impact on whether a company eventually survives or closes.
Key Findings
Among the factors regarding the ability to survive of starting up companies, the factors such as the number of employees, funding total amount, and the active days have significant influences on the company’s survivability. On the contrary, the factors such as country, region or number of funding rounds do not have significant influences. Whether a company acts as an investor simultaneously will also have influences on whether the company will become an IPO or will be acquired.
Future Work
Our future work is expected to integrate the inspirative insight gained from our case study with the methodology of our own. Three items are listed as in the following:
- Construct a heterogeneous relationship network for survival rate prediction [5].
- Define a data path score according to HeteSim algorithm [6][7].
- Predict company survival rate using MLP, decision tree and other neural networks.
- Predict how much money a company will raise.
References
- Bill Murphy , The Start-up Hall of Shame (America’s 10 Worst States for Entrepreneurs), © 2018 Manuseto Ventures, inc.com/bill-murphy-jr/the-startup-hall-of-shame-americas-10-worst-states-for-entrepreneurs.html
- Bill Murphy , 10 Top States for Entrepreneurship and Innovation, © 2018 Manuseto Ventures, inc.com/bill-murphy-jr/ranking-the-10-top-states-for-entrepreneurship-and-innovation.html
- Enterprising States: States Innovate, © 2015 The U.S. Chamber of Commerce Foundation, www.uschamberfoundation.org/enterprisingstates/
assets/files/Executive-Summary-OL.pdf - Zameena Mejia, The top 10 best countries for entrepreneurs in 2018, © 2019 CNBC LLC,
https://www.cnbc.com/2018/02/05/
us-world-news-report-2018-top-10-best-countries-for-entrepreneurs.html - Xiangxiang Zeng, You Li, Stephen C.H. Leung, Ziyu Lin, Xiangrong Liu, Investment behavior prediction in heterogeneous information network, Neurocomputing, Volume 217, 2016, Pages 125-132
- Sun, Y., & Han, J. (2012). Mining Heterogeneous Information Networks: Principles and Methodologies. Synthesis Lectures on Data Mining and Knowledge Discovery, 3(2), 1-159
- Shi, C., Kong, X., Huang, Y., Yu, P. S., & Wu, B. (2014). HeteSim: A General Framework for Relevance Measure in Heterogeneous Networks. IEEE Transactions on Knowledge and Data Engineering, 26(10), 2479-2492. [6702458].