Replace text in big file using sedstring substituton for match patternDiffing two big text filesReplace some...

How does a simple logistic regression model achieve a 92% classification accuracy on MNIST?

Ambiguity in notation resolved by +

Were Roman public roads build by private companies?

Why do sellers care about down payments?

How to prepare for STEM graduate program while in industry?

Why some files are not movable in Windows 10

What organs or modifications would be needed for a life biological creature not to require sleep?

Does my opponent need to prove his creature has morph?

Mutable named tuple with default value and conditional rounding support

What are uses of the byte after BRK instruction on 6502?

I was promised a work PC but still awaiting approval 3 months later so using my own laptop - Is it fair to ask employer for laptop insurance?

Can I tap all my opponent's lands while they're casting a spell to negate it?

What is the mathematical notation for rounding a given number to the nearest integer?

How do certain apps show new notifications when internet access is restricted to them?

Speedometer as a symbol into awesomebox

How can I discourage sharing internal API keys within a company?

What officially disallows US presidents from driving?

"Literally" Vs "In the true sense of the word"

Can derivatives be defined as anti-integrals?

In what state are satellites left in when they are left in a graveyard orbit?

Is the Dodge action perceptible to other characters?

Output a Super Mario Image

Write a function that returns an iterable object of all valid points 4-directionally adjacent to (x, y)

How to stabilise the bicycle seatpost and saddle when it is all the way up?



Replace text in big file using sed


string substituton for match patternDiffing two big text filesReplace some text with sedReplace a range of text with special characters using sedSed: replace text keeping digit occurrenceReplace text quickly in very large fileNot able to replace text using sedreplace/remove text (sed) by a fileReplace *n.*n= using sed






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}







0















I have a big file that contains a text will be replaced based on a regular expression passed to.
The regular expression is tested and can find matching pattern, but when I use it with sed no text replaced.
Note that my environment is Windows, and I have 16GB memory, file size is ~14GB.



sed -i "/,[rn]+  CONSTRAINT `[a-zA-Z0-9_]+` FOREIGN KEY (`[a-zA-Z0-9_]+`) REFERENCES `[a-zA-Z0-9_]+` (`[a-zA-Z0-9_]+`)/ s// /g" all_files_test.sql


all_files_test.sql file:



DROP TABLE IF EXISTS `holdings_FLZWHX`;
;
;
CREATE TABLE `holdings_FLZWHX` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`DateAdded` datetime NOT NULL,
`FundId` int(11) NOT NULL,
`AssetId` int(11) NOT NULL,
`DayChangeEqt` decimal(18,2) DEFAULT NULL,
`PMinDayRet` decimal(18,2) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `holdings_FLZWHX_FundId` (`FundId`),
KEY `holdings_FLZWHX_AssetId` (`AssetId`),
KEY `holdings_FLZWHX_TypeId` (`TypeId`),
KEY `holdings_FLZWHX_TickerId` (`TickerId`),
KEY `holdings_FLZWHX_StyleId` (`StyleId`),
KEY `holdings_FLZWHX_SectorId` (`SectorId`),
KEY `holdings_FLZWHX_CountryId` (`CountryId`),
KEY `holdings_FLZWHX_CurrencyId` (`CurrencyId`),
CONSTRAINT `holdings_FLZWHX_ibfk_1` FOREIGN KEY (`FundId`) REFERENCES `target_funds` (`fundid`),
CONSTRAINT `holdings_FLZWHX_ibfk_2` FOREIGN KEY (`AssetId`) REFERENCES `asset` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_3` FOREIGN KEY (`TypeId`) REFERENCES `type` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_4` FOREIGN KEY (`TickerId`) REFERENCES `ticker` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_5` FOREIGN KEY (`StyleId`) REFERENCES `style` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_6` FOREIGN KEY (`SectorId`) REFERENCES `sector` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_7` FOREIGN KEY (`CountryId`) REFERENCES `country` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_8` FOREIGN KEY (`CurrencyId`) REFERENCES `currency` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4201 DEFAULT CHARSET=latin1;
;


The result I was expecting:



DROP TABLE IF EXISTS `holdings_FLZWHX`;
;
;
CREATE TABLE `holdings_FLZWHX` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`DateAdded` datetime NOT NULL,
`FundId` int(11) NOT NULL,
`AssetId` int(11) NOT NULL,
`DayChangeEqt` decimal(18,2) DEFAULT NULL,
`PMinDayRet` decimal(18,2) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4201 DEFAULT CHARSET=latin1;
;









share|improve this question









New contributor



H Aßdøµ is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






















  • @msp9011 I added the expected result.

    – H Aßdøµ
    26 mins ago











  • try sed '/CONSTRAINT/d;/ KEY/d' all_files_test.sql

    – msp9011
    21 mins ago













  • unix.stackexchange.com/questions/532869/… Check this link. It's because of the quotes.

    – rahuls36
    19 mins ago











  • But I need the full pattern I set.

    – H Aßdøµ
    14 mins ago











  • Note that [rn]+ matches on either a backslash r or n followed by a +. sed regexps are basic regexps by default. In any case, sed works on one line at a time so what it matches on will never contain a newline character.

    – Stéphane Chazelas
    3 mins ago


















0















I have a big file that contains a text will be replaced based on a regular expression passed to.
The regular expression is tested and can find matching pattern, but when I use it with sed no text replaced.
Note that my environment is Windows, and I have 16GB memory, file size is ~14GB.



sed -i "/,[rn]+  CONSTRAINT `[a-zA-Z0-9_]+` FOREIGN KEY (`[a-zA-Z0-9_]+`) REFERENCES `[a-zA-Z0-9_]+` (`[a-zA-Z0-9_]+`)/ s// /g" all_files_test.sql


all_files_test.sql file:



DROP TABLE IF EXISTS `holdings_FLZWHX`;
;
;
CREATE TABLE `holdings_FLZWHX` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`DateAdded` datetime NOT NULL,
`FundId` int(11) NOT NULL,
`AssetId` int(11) NOT NULL,
`DayChangeEqt` decimal(18,2) DEFAULT NULL,
`PMinDayRet` decimal(18,2) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `holdings_FLZWHX_FundId` (`FundId`),
KEY `holdings_FLZWHX_AssetId` (`AssetId`),
KEY `holdings_FLZWHX_TypeId` (`TypeId`),
KEY `holdings_FLZWHX_TickerId` (`TickerId`),
KEY `holdings_FLZWHX_StyleId` (`StyleId`),
KEY `holdings_FLZWHX_SectorId` (`SectorId`),
KEY `holdings_FLZWHX_CountryId` (`CountryId`),
KEY `holdings_FLZWHX_CurrencyId` (`CurrencyId`),
CONSTRAINT `holdings_FLZWHX_ibfk_1` FOREIGN KEY (`FundId`) REFERENCES `target_funds` (`fundid`),
CONSTRAINT `holdings_FLZWHX_ibfk_2` FOREIGN KEY (`AssetId`) REFERENCES `asset` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_3` FOREIGN KEY (`TypeId`) REFERENCES `type` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_4` FOREIGN KEY (`TickerId`) REFERENCES `ticker` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_5` FOREIGN KEY (`StyleId`) REFERENCES `style` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_6` FOREIGN KEY (`SectorId`) REFERENCES `sector` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_7` FOREIGN KEY (`CountryId`) REFERENCES `country` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_8` FOREIGN KEY (`CurrencyId`) REFERENCES `currency` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4201 DEFAULT CHARSET=latin1;
;


The result I was expecting:



DROP TABLE IF EXISTS `holdings_FLZWHX`;
;
;
CREATE TABLE `holdings_FLZWHX` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`DateAdded` datetime NOT NULL,
`FundId` int(11) NOT NULL,
`AssetId` int(11) NOT NULL,
`DayChangeEqt` decimal(18,2) DEFAULT NULL,
`PMinDayRet` decimal(18,2) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4201 DEFAULT CHARSET=latin1;
;









share|improve this question









New contributor



H Aßdøµ is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






















  • @msp9011 I added the expected result.

    – H Aßdøµ
    26 mins ago











  • try sed '/CONSTRAINT/d;/ KEY/d' all_files_test.sql

    – msp9011
    21 mins ago













  • unix.stackexchange.com/questions/532869/… Check this link. It's because of the quotes.

    – rahuls36
    19 mins ago











  • But I need the full pattern I set.

    – H Aßdøµ
    14 mins ago











  • Note that [rn]+ matches on either a backslash r or n followed by a +. sed regexps are basic regexps by default. In any case, sed works on one line at a time so what it matches on will never contain a newline character.

    – Stéphane Chazelas
    3 mins ago














0












0








0








I have a big file that contains a text will be replaced based on a regular expression passed to.
The regular expression is tested and can find matching pattern, but when I use it with sed no text replaced.
Note that my environment is Windows, and I have 16GB memory, file size is ~14GB.



sed -i "/,[rn]+  CONSTRAINT `[a-zA-Z0-9_]+` FOREIGN KEY (`[a-zA-Z0-9_]+`) REFERENCES `[a-zA-Z0-9_]+` (`[a-zA-Z0-9_]+`)/ s// /g" all_files_test.sql


all_files_test.sql file:



DROP TABLE IF EXISTS `holdings_FLZWHX`;
;
;
CREATE TABLE `holdings_FLZWHX` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`DateAdded` datetime NOT NULL,
`FundId` int(11) NOT NULL,
`AssetId` int(11) NOT NULL,
`DayChangeEqt` decimal(18,2) DEFAULT NULL,
`PMinDayRet` decimal(18,2) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `holdings_FLZWHX_FundId` (`FundId`),
KEY `holdings_FLZWHX_AssetId` (`AssetId`),
KEY `holdings_FLZWHX_TypeId` (`TypeId`),
KEY `holdings_FLZWHX_TickerId` (`TickerId`),
KEY `holdings_FLZWHX_StyleId` (`StyleId`),
KEY `holdings_FLZWHX_SectorId` (`SectorId`),
KEY `holdings_FLZWHX_CountryId` (`CountryId`),
KEY `holdings_FLZWHX_CurrencyId` (`CurrencyId`),
CONSTRAINT `holdings_FLZWHX_ibfk_1` FOREIGN KEY (`FundId`) REFERENCES `target_funds` (`fundid`),
CONSTRAINT `holdings_FLZWHX_ibfk_2` FOREIGN KEY (`AssetId`) REFERENCES `asset` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_3` FOREIGN KEY (`TypeId`) REFERENCES `type` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_4` FOREIGN KEY (`TickerId`) REFERENCES `ticker` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_5` FOREIGN KEY (`StyleId`) REFERENCES `style` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_6` FOREIGN KEY (`SectorId`) REFERENCES `sector` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_7` FOREIGN KEY (`CountryId`) REFERENCES `country` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_8` FOREIGN KEY (`CurrencyId`) REFERENCES `currency` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4201 DEFAULT CHARSET=latin1;
;


The result I was expecting:



DROP TABLE IF EXISTS `holdings_FLZWHX`;
;
;
CREATE TABLE `holdings_FLZWHX` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`DateAdded` datetime NOT NULL,
`FundId` int(11) NOT NULL,
`AssetId` int(11) NOT NULL,
`DayChangeEqt` decimal(18,2) DEFAULT NULL,
`PMinDayRet` decimal(18,2) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4201 DEFAULT CHARSET=latin1;
;









share|improve this question









New contributor



H Aßdøµ is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











I have a big file that contains a text will be replaced based on a regular expression passed to.
The regular expression is tested and can find matching pattern, but when I use it with sed no text replaced.
Note that my environment is Windows, and I have 16GB memory, file size is ~14GB.



sed -i "/,[rn]+  CONSTRAINT `[a-zA-Z0-9_]+` FOREIGN KEY (`[a-zA-Z0-9_]+`) REFERENCES `[a-zA-Z0-9_]+` (`[a-zA-Z0-9_]+`)/ s// /g" all_files_test.sql


all_files_test.sql file:



DROP TABLE IF EXISTS `holdings_FLZWHX`;
;
;
CREATE TABLE `holdings_FLZWHX` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`DateAdded` datetime NOT NULL,
`FundId` int(11) NOT NULL,
`AssetId` int(11) NOT NULL,
`DayChangeEqt` decimal(18,2) DEFAULT NULL,
`PMinDayRet` decimal(18,2) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `holdings_FLZWHX_FundId` (`FundId`),
KEY `holdings_FLZWHX_AssetId` (`AssetId`),
KEY `holdings_FLZWHX_TypeId` (`TypeId`),
KEY `holdings_FLZWHX_TickerId` (`TickerId`),
KEY `holdings_FLZWHX_StyleId` (`StyleId`),
KEY `holdings_FLZWHX_SectorId` (`SectorId`),
KEY `holdings_FLZWHX_CountryId` (`CountryId`),
KEY `holdings_FLZWHX_CurrencyId` (`CurrencyId`),
CONSTRAINT `holdings_FLZWHX_ibfk_1` FOREIGN KEY (`FundId`) REFERENCES `target_funds` (`fundid`),
CONSTRAINT `holdings_FLZWHX_ibfk_2` FOREIGN KEY (`AssetId`) REFERENCES `asset` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_3` FOREIGN KEY (`TypeId`) REFERENCES `type` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_4` FOREIGN KEY (`TickerId`) REFERENCES `ticker` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_5` FOREIGN KEY (`StyleId`) REFERENCES `style` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_6` FOREIGN KEY (`SectorId`) REFERENCES `sector` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_7` FOREIGN KEY (`CountryId`) REFERENCES `country` (`id`),
CONSTRAINT `holdings_FLZWHX_ibfk_8` FOREIGN KEY (`CurrencyId`) REFERENCES `currency` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4201 DEFAULT CHARSET=latin1;
;


The result I was expecting:



DROP TABLE IF EXISTS `holdings_FLZWHX`;
;
;
CREATE TABLE `holdings_FLZWHX` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`DateAdded` datetime NOT NULL,
`FundId` int(11) NOT NULL,
`AssetId` int(11) NOT NULL,
`DayChangeEqt` decimal(18,2) DEFAULT NULL,
`PMinDayRet` decimal(18,2) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4201 DEFAULT CHARSET=latin1;
;






sed regular-expression large-files






share|improve this question









New contributor



H Aßdøµ is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.










share|improve this question









New contributor



H Aßdøµ is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.








share|improve this question




share|improve this question








edited 26 mins ago







H Aßdøµ













New contributor



H Aßdøµ is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.








asked 42 mins ago









H AßdøµH Aßdøµ

1013 bronze badges




1013 bronze badges




New contributor



H Aßdøµ is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




New contributor




H Aßdøµ is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.


















  • @msp9011 I added the expected result.

    – H Aßdøµ
    26 mins ago











  • try sed '/CONSTRAINT/d;/ KEY/d' all_files_test.sql

    – msp9011
    21 mins ago













  • unix.stackexchange.com/questions/532869/… Check this link. It's because of the quotes.

    – rahuls36
    19 mins ago











  • But I need the full pattern I set.

    – H Aßdøµ
    14 mins ago











  • Note that [rn]+ matches on either a backslash r or n followed by a +. sed regexps are basic regexps by default. In any case, sed works on one line at a time so what it matches on will never contain a newline character.

    – Stéphane Chazelas
    3 mins ago



















  • @msp9011 I added the expected result.

    – H Aßdøµ
    26 mins ago











  • try sed '/CONSTRAINT/d;/ KEY/d' all_files_test.sql

    – msp9011
    21 mins ago













  • unix.stackexchange.com/questions/532869/… Check this link. It's because of the quotes.

    – rahuls36
    19 mins ago











  • But I need the full pattern I set.

    – H Aßdøµ
    14 mins ago











  • Note that [rn]+ matches on either a backslash r or n followed by a +. sed regexps are basic regexps by default. In any case, sed works on one line at a time so what it matches on will never contain a newline character.

    – Stéphane Chazelas
    3 mins ago

















@msp9011 I added the expected result.

– H Aßdøµ
26 mins ago





@msp9011 I added the expected result.

– H Aßdøµ
26 mins ago













try sed '/CONSTRAINT/d;/ KEY/d' all_files_test.sql

– msp9011
21 mins ago







try sed '/CONSTRAINT/d;/ KEY/d' all_files_test.sql

– msp9011
21 mins ago















unix.stackexchange.com/questions/532869/… Check this link. It's because of the quotes.

– rahuls36
19 mins ago





unix.stackexchange.com/questions/532869/… Check this link. It's because of the quotes.

– rahuls36
19 mins ago













But I need the full pattern I set.

– H Aßdøµ
14 mins ago





But I need the full pattern I set.

– H Aßdøµ
14 mins ago













Note that [rn]+ matches on either a backslash r or n followed by a +. sed regexps are basic regexps by default. In any case, sed works on one line at a time so what it matches on will never contain a newline character.

– Stéphane Chazelas
3 mins ago





Note that [rn]+ matches on either a backslash r or n followed by a +. sed regexps are basic regexps by default. In any case, sed works on one line at a time so what it matches on will never contain a newline character.

– Stéphane Chazelas
3 mins ago










0






active

oldest

votes














Your Answer








StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/4.0/"u003ecc by-sa 4.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});







H Aßdøµ is a new contributor. Be nice, and check out our Code of Conduct.










draft saved

draft discarded
















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f541516%2freplace-text-in-big-file-using-sed%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes









H Aßdøµ is a new contributor. Be nice, and check out our Code of Conduct.










draft saved

draft discarded

















H Aßdøµ is a new contributor. Be nice, and check out our Code of Conduct.













H Aßdøµ is a new contributor. Be nice, and check out our Code of Conduct.












H Aßdøµ is a new contributor. Be nice, and check out our Code of Conduct.
















Thanks for contributing an answer to Unix & Linux Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f541516%2freplace-text-in-big-file-using-sed%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Taj Mahal Inhaltsverzeichnis Aufbau | Geschichte | 350-Jahr-Feier | Heutige Bedeutung | Siehe auch |...

Baia Sprie Cuprins Etimologie | Istorie | Demografie | Politică și administrație | Arii naturale...

Nicolae Petrescu-Găină Cuprins Biografie | Opera | In memoriam | Varia | Controverse, incertitudini...