Mdthbs

How to deal with employer who keeps me at work after working hours

Why are the capacitors necessary for a quartz crystal?

Is there a word that describes the unjustified use of a more complex word?

What's the 2-minute timer on mobile Deutsche Bahn tickets?

Hostile Divisor Numbers

How to properly store the current value of int variable into a token list?

Make me a minimum magic sum

Why would one crossvalidate the random state number?

Why is my arithmetic with a long long int behaving this way?

In linear regression why does regularisation penalise the parameter values as well?

Is there an age requirement to play in Adventurers League?

In Futurama, how many beings has Leela slept with?

Is there a proof that the set of real numbers can exactly represent distances?

My first C++ game (snake console game)

Has the Hulk always been able to talk?

All of my Firefox add-ons been disabled suddenly, how can I re-enable them?

What happens if I accidentally leave an app running and click "Install Now" in Software Updater?

How do I allocate more memory to an app on Sheepshaver running Mac OS 9?

Can I hide the part of long lines that exceeds the visual line?

Which US defense organization would respond to an invasion like this?

How to remap repeating commands i.e. <number><command>?

How can a hefty sand storm happen in a thin atmosphere like Martian?

Krull dimension of the ring of global sections

Can my 2 children, aged 10 and 12, who are US citizens, travel to the USA on expired American passports?

Why would one crossvalidate the random state number?

Linear kernel in SVM performing much worse than RBF or PolyWhy is the number of samples smaller than the number of values in my decision tree?How does one fine-tune parameters and weights at the same time?Predicting contract churn/cancellation: Great model results does not work in the real worldWhy is this Random Forest perfect?Why would a fake feature with random numbers get selected in feature importance?Random state in machine learning modelsIs a good shuffle random state for training data really good for the model?Why is the reported loss different from the mean squared error calculated on the train data?Why is my MLP with 2 features is doing worse than MLP with 1 feature where the one feature is a combination of feature1*feature2?

2

$begingroup$

Still learning about machine learning, I've stumbled across a kaggle (link) which I cannot understand.

Here are the lines 72 and 73:

parameters = {'solver': ['lbfgs'], 

              'max_iter': [1000,1100,1200,1300,1400,1500,1600,1700,1800,1900,2000 ], 

              'alpha': 10.0 ** -np.arange(1, 10), 

              'hidden_layer_sizes':np.arange(10, 15), 

              'random_state':[0,1,2,3,4,5,6,7,8,9]}

clf = GridSearchCV(MLPClassifier(), parameters, n_jobs=-1)

As you can see, the random_state parameter is been tested across 10 values.

What is the point of doing this?

If one model perform better with some random_state, does it make any sense to use this particular parameter on other models?

scikit-learn mlp

share|improve this question

asked 4 hours ago

Dan ChaltielDan Chaltiel

1757

$endgroup$

add a comment | 

2

$begingroup$

Still learning about machine learning, I've stumbled across a kaggle (link) which I cannot understand.

Here are the lines 72 and 73:

parameters = {'solver': ['lbfgs'], 

              'max_iter': [1000,1100,1200,1300,1400,1500,1600,1700,1800,1900,2000 ], 

              'alpha': 10.0 ** -np.arange(1, 10), 

              'hidden_layer_sizes':np.arange(10, 15), 

              'random_state':[0,1,2,3,4,5,6,7,8,9]}

clf = GridSearchCV(MLPClassifier(), parameters, n_jobs=-1)

As you can see, the random_state parameter is been tested across 10 values.

What is the point of doing this?

If one model perform better with some random_state, does it make any sense to use this particular parameter on other models?

scikit-learn mlp

share|improve this question

asked 4 hours ago

Dan ChaltielDan Chaltiel

1757

$endgroup$

add a comment | 

$begingroup$

Still learning about machine learning, I've stumbled across a kaggle (link) which I cannot understand.

Here are the lines 72 and 73:

parameters = {'solver': ['lbfgs'], 

              'max_iter': [1000,1100,1200,1300,1400,1500,1600,1700,1800,1900,2000 ], 

              'alpha': 10.0 ** -np.arange(1, 10), 

              'hidden_layer_sizes':np.arange(10, 15), 

              'random_state':[0,1,2,3,4,5,6,7,8,9]}

clf = GridSearchCV(MLPClassifier(), parameters, n_jobs=-1)

As you can see, the random_state parameter is been tested across 10 values.

What is the point of doing this?

If one model perform better with some random_state, does it make any sense to use this particular parameter on other models?

scikit-learn mlp

share|improve this question

asked 4 hours ago

Dan ChaltielDan Chaltiel

1757

$endgroup$

parameters = {'solver': ['lbfgs'], 

              'max_iter': [1000,1100,1200,1300,1400,1500,1600,1700,1800,1900,2000 ], 

              'alpha': 10.0 ** -np.arange(1, 10), 

              'hidden_layer_sizes':np.arange(10, 15), 

              'random_state':[0,1,2,3,4,5,6,7,8,9]}

clf = GridSearchCV(MLPClassifier(), parameters, n_jobs=-1)

share|improve this question

asked 4 hours ago

Dan ChaltielDan Chaltiel

1757

share|improve this question

asked 4 hours ago

Dan ChaltielDan Chaltiel

1757

asked 4 hours ago

Dan ChaltielDan Chaltiel

1757

asked 4 hours ago

Dan ChaltielDan Chaltiel

1757

1 Answer
1

active

oldest

votes

3

$begingroup$

I personally think that the general idea of optimising your model with different random seeds is not a good idea. There are many other, more important, aspects of the modelling process that you can worry about, tweak and compare before spending time on the effects of random initialisation.

That being said, if you just want to test the effect of random initialisation of model weights on a final validation metric, this could be an approach to do so. Kind of the reverse argument to my point above. If you can show for different random seeds (ceteris paribus: with all other parameters equal) that the final model performs differently, it shows maybe that their is either inconsistency in the model, or a bug in the code even. I would not expect a well-validated model to give hugely differing results if being run with a different random seed, so if it does, it tells me something weird is going on!

share|improve this answer

answered 4 hours ago

n1k31t4n1k31t4

6,9462422

$endgroup$

add a comment | 

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f51397%2fwhy-would-one-crossvalidate-the-random-state-number%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

3

$begingroup$

I personally think that the general idea of optimising your model with different random seeds is not a good idea. There are many other, more important, aspects of the modelling process that you can worry about, tweak and compare before spending time on the effects of random initialisation.

That being said, if you just want to test the effect of random initialisation of model weights on a final validation metric, this could be an approach to do so. Kind of the reverse argument to my point above. If you can show for different random seeds (ceteris paribus: with all other parameters equal) that the final model performs differently, it shows maybe that their is either inconsistency in the model, or a bug in the code even. I would not expect a well-validated model to give hugely differing results if being run with a different random seed, so if it does, it tells me something weird is going on!

share|improve this answer

answered 4 hours ago

n1k31t4n1k31t4

6,9462422

$endgroup$

add a comment | 

3

$begingroup$

I personally think that the general idea of optimising your model with different random seeds is not a good idea. There are many other, more important, aspects of the modelling process that you can worry about, tweak and compare before spending time on the effects of random initialisation.

That being said, if you just want to test the effect of random initialisation of model weights on a final validation metric, this could be an approach to do so. Kind of the reverse argument to my point above. If you can show for different random seeds (ceteris paribus: with all other parameters equal) that the final model performs differently, it shows maybe that their is either inconsistency in the model, or a bug in the code even. I would not expect a well-validated model to give hugely differing results if being run with a different random seed, so if it does, it tells me something weird is going on!

share|improve this answer

answered 4 hours ago

n1k31t4n1k31t4

6,9462422

$endgroup$

add a comment | 

$begingroup$

I personally think that the general idea of optimising your model with different random seeds is not a good idea. There are many other, more important, aspects of the modelling process that you can worry about, tweak and compare before spending time on the effects of random initialisation.

That being said, if you just want to test the effect of random initialisation of model weights on a final validation metric, this could be an approach to do so. Kind of the reverse argument to my point above. If you can show for different random seeds (ceteris paribus: with all other parameters equal) that the final model performs differently, it shows maybe that their is either inconsistency in the model, or a bug in the code even. I would not expect a well-validated model to give hugely differing results if being run with a different random seed, so if it does, it tells me something weird is going on!

share|improve this answer

answered 4 hours ago

n1k31t4n1k31t4

6,9462422

$endgroup$

share|improve this answer

answered 4 hours ago

n1k31t4n1k31t4

6,9462422

answered 4 hours ago

n1k31t4n1k31t4

6,9462422

answered 4 hours ago

n1k31t4n1k31t4

6,9462422