I have the following parameters set up :

parameter_space = { 'hidden_layer_sizes': [(sp_randint(100,600),sp_randint(100,600),), (sp_randint(100,600),)], 'activation': ['tanh', 'relu', 'logistic'], 'solver': ['sgd', 'adam', 'lbfgs'], 'alpha': stats.uniform(0.0001, 0.9), 'learning_rate': ['constant','adaptive']} 

All the parameters except the hidden_layer_sizes is working as expected. However, fitting this RandomizedSearchCV model and displaying it's verbose text shows that it treats hidden_layer_sizes as :

hidden_layer_sizes=(<scipy.stats._distn_infrastructure.rv_frozen object and goes on to throw : TypeError: '<=' not supported between instances of 'rv_frozen' and 'int'

This result is obtained instead of the expected 1 or 2 layer MLP with hidden layer neurons between 100 and 600. Any ideas / other related tips?

1 Answer

sp_randint returns an instance of the rv_discrete class / a randint object, in order to generate random numbers, the correct syntax should be sp_randint.rvs(low, high, size). In order to make it work you need to define the parameter_space as below:

parameter_space = { 'hidden_layer_sizes': [(sp_randint.rvs(100,600,1),sp_randint.rvs(100,600,1),), (sp_randint.rvs(100,600,1),)], 'activation': ['tanh', 'relu', 'logistic'], 'solver': ['sgd', 'adam', 'lbfgs'], 'alpha': uniform(0.0001, 0.9), 'learning_rate': ['constant','adaptive']}

ncG1vNJzZmirpJawrLvVnqmfpJ%2Bse6S7zGiorp2jqbawutJobGtoYm6BcYSOrKKlnZGnu265y6lknKSRqMCqssieqWagqaWys7zAq5imnaSav267z62gpqGqlsGqu81mqZqmlKS6qsbEnaqemaKYtaTC