Mdthbs

The No-Free-Lunch Theorem and K-NN consistency

What verb is かまされる?

How would a Creature that needs to be seen by Humans evolve?

Why is the UK so keen to remove the "backstop" when their leadership seems to think that no border will be needed in Northern Ireland?

Was the Boeing 2707 design flawed?

What would make bones be of different colors?

Can RMSE and MAE have the same value?

Read file lines into shell line separated by space

Was there ever a treaty between 2 entities with significantly different translations to the detriment of one party?

Could George I (of Great Britain) speak English?

What is the difference between "Grippe" and "Männergrippe"?

Improving Performance of an XY Monte Carlo

Sum ergo cogito?

Are the players on the same team as the DM?

Duplicate Files

How do we calculate energy of food?

Why do all fields in a QFT transform like *irreducible* representations of some group?

Is there any way to keep a player from killing an NPC?

Network helper class with retry logic on failure

Is "The life is beautiful" incorrect or just very non-idiomatic?

How can I unambiguously ask for a new user's "Display Name"?

Where can/should I, as a high schooler, publish a paper regarding the derivation of a formula?

How to find out the average duration of the peer-review process for a given journal?

Can I get temporary health insurance while moving to the US?

Copying files from specific subsubdirectories based on subdirectory name into single directory then renaming

How to have find only search for files in changed directories?Use find + sed + cp to find files and copy them to a directory with a different nameFind Directory and copy another directory to found directory?Copy files based on date/time from subset of directoriesCopy with Find Creates Duplicate FilesRemoving names of pruned directories from output of find commandlist directory recursively, with subpath, and leaf nodes (files) first (for batch renaming part of filenames)?find files with particular attributes, rename them to their directory name, copy them elsewhere

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}

0

So here's an excerpt of my directory tree:

|-- 20070214_014700.a  

|   |-- info  

|   |-- processed  

|       |-- HH.EL..BHZ  

|       |-- AZ.AS..HHZ  

|       |-- (hundreds more)  

|   |-- raw  

|   |-- resp  

|-- 20100737_055560.a  

|   |-- info  

|   |-- processed  

|   |-- raw  

|   |-- resp  

|-- 20190537_028750.a  

|   |-- info  

|   |-- processed  

|   |-- raw  

|   |-- resp  

I have ~13,000 directories (ending in .a) and each directory has a 'processed' subdirectory which has files I'd like to copy from every processed/ directory into a single directory. Some of these files may have the same filename so I'd also like to rename them based on their parent directory. I'm not too picky but something similar to:

20070214_014700_HH.EL..BHZ

The whole dataset is 3 TB so I've been testing on just a few directories using 'find':

find . -name processed -exec cp -r '{}' 'test/{}' ;

For some reason this dumps some files into test/ but also creates another processed/ directory inside of that. I'm not sure how to include a copy command and renaming function into find at the same time so any advice would be great. Thanks for the help.

command-line files find tree

share|improve this question

asked yesterday

LividNoodleLividNoodle

311 bronze badge

New contributor

LividNoodle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

add a comment | 

0

So here's an excerpt of my directory tree:

|-- 20070214_014700.a  

|   |-- info  

|   |-- processed  

|       |-- HH.EL..BHZ  

|       |-- AZ.AS..HHZ  

|       |-- (hundreds more)  

|   |-- raw  

|   |-- resp  

|-- 20100737_055560.a  

|   |-- info  

|   |-- processed  

|   |-- raw  

|   |-- resp  

|-- 20190537_028750.a  

|   |-- info  

|   |-- processed  

|   |-- raw  

|   |-- resp  

I have ~13,000 directories (ending in .a) and each directory has a 'processed' subdirectory which has files I'd like to copy from every processed/ directory into a single directory. Some of these files may have the same filename so I'd also like to rename them based on their parent directory. I'm not too picky but something similar to:

20070214_014700_HH.EL..BHZ

The whole dataset is 3 TB so I've been testing on just a few directories using 'find':

find . -name processed -exec cp -r '{}' 'test/{}' ;

For some reason this dumps some files into test/ but also creates another processed/ directory inside of that. I'm not sure how to include a copy command and renaming function into find at the same time so any advice would be great. Thanks for the help.

command-line files find tree

share|improve this question

asked yesterday

LividNoodleLividNoodle

311 bronze badge

New contributor

LividNoodle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

add a comment | 

So here's an excerpt of my directory tree:

|-- 20070214_014700.a  

|   |-- info  

|   |-- processed  

|       |-- HH.EL..BHZ  

|       |-- AZ.AS..HHZ  

|       |-- (hundreds more)  

|   |-- raw  

|   |-- resp  

|-- 20100737_055560.a  

|   |-- info  

|   |-- processed  

|   |-- raw  

|   |-- resp  

|-- 20190537_028750.a  

|   |-- info  

|   |-- processed  

|   |-- raw  

|   |-- resp  

I have ~13,000 directories (ending in .a) and each directory has a 'processed' subdirectory which has files I'd like to copy from every processed/ directory into a single directory. Some of these files may have the same filename so I'd also like to rename them based on their parent directory. I'm not too picky but something similar to:

20070214_014700_HH.EL..BHZ

The whole dataset is 3 TB so I've been testing on just a few directories using 'find':

find . -name processed -exec cp -r '{}' 'test/{}' ;

For some reason this dumps some files into test/ but also creates another processed/ directory inside of that. I'm not sure how to include a copy command and renaming function into find at the same time so any advice would be great. Thanks for the help.

command-line files find tree

share|improve this question

asked yesterday

LividNoodleLividNoodle

311 bronze badge

New contributor

LividNoodle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

|-- 20070214_014700.a  

|   |-- info  

|   |-- processed  

|       |-- HH.EL..BHZ  

|       |-- AZ.AS..HHZ  

|       |-- (hundreds more)  

|   |-- raw  

|   |-- resp  

|-- 20100737_055560.a  

|   |-- info  

|   |-- processed  

|   |-- raw  

|   |-- resp  

|-- 20190537_028750.a  

|   |-- info  

|   |-- processed  

|   |-- raw  

|   |-- resp  

20070214_014700_HH.EL..BHZ

find . -name processed -exec cp -r '{}' 'test/{}' ;

share|improve this question

asked yesterday

LividNoodleLividNoodle

311 bronze badge

New contributor

LividNoodle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|improve this question

asked yesterday

LividNoodleLividNoodle

311 bronze badge

New contributor

LividNoodle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked yesterday

LividNoodleLividNoodle

311 bronze badge

New contributor

LividNoodle is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked yesterday

LividNoodleLividNoodle

311 bronze badge

1 Answer
1

active

oldest

votes

0

find . -type f -path "./*.a/processed/*" -exec sh -c '

for path; do

  prefix=${path%%.a/processed*}

  cp "$path" "test/${prefix##*/}_${path##*processed/}"

done

' sh {} +

Option -type f searches for regular files in the given path and the -exec option starts a shell script with find's result as arguments ({} +).
In the for-loop each argument is assigned to the path variable.

Example: If variable path is ./20070214_014700.a/processed/AZ.AS..HHZ, then

• 
  prefix=${path%%.a/processed*} removes the suffix -> ./20070214_014700
  


• 
  ${prefix##*/} removes the prefix to the first / -> 20070214_014700
  


• 
  ${path##*processed/} also removes the prefix and leaves the filename -> AZ.AS..HHZ
  

The resulting target filename of the cp command is test/20070214_014700_AZ.AS..HHZ.

share|improve this answer

answered yesterday

FreddyFreddy

6,66011 gold badge66 silver badges2424 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Perfect, tested it out and it fits the bill exactly. Thanks a lot for the help, just starting to learn the command line so your example is very educational.
  
  – LividNoodle
  yesterday
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

LividNoodle is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f537163%2fcopying-files-from-specific-subsubdirectories-based-on-subdirectory-name-into-si%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

0

find . -type f -path "./*.a/processed/*" -exec sh -c '

for path; do

  prefix=${path%%.a/processed*}

  cp "$path" "test/${prefix##*/}_${path##*processed/}"

done

' sh {} +

Option -type f searches for regular files in the given path and the -exec option starts a shell script with find's result as arguments ({} +).
In the for-loop each argument is assigned to the path variable.

Example: If variable path is ./20070214_014700.a/processed/AZ.AS..HHZ, then

• 
  prefix=${path%%.a/processed*} removes the suffix -> ./20070214_014700
  


• 
  ${prefix##*/} removes the prefix to the first / -> 20070214_014700
  


• 
  ${path##*processed/} also removes the prefix and leaves the filename -> AZ.AS..HHZ
  

The resulting target filename of the cp command is test/20070214_014700_AZ.AS..HHZ.

share|improve this answer

answered yesterday

FreddyFreddy

6,66011 gold badge66 silver badges2424 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Perfect, tested it out and it fits the bill exactly. Thanks a lot for the help, just starting to learn the command line so your example is very educational.
  
  – LividNoodle
  yesterday
  

  
  
  
  

add a comment | 

0

find . -type f -path "./*.a/processed/*" -exec sh -c '

for path; do

  prefix=${path%%.a/processed*}

  cp "$path" "test/${prefix##*/}_${path##*processed/}"

done

' sh {} +

Option -type f searches for regular files in the given path and the -exec option starts a shell script with find's result as arguments ({} +).
In the for-loop each argument is assigned to the path variable.

Example: If variable path is ./20070214_014700.a/processed/AZ.AS..HHZ, then

• 
  prefix=${path%%.a/processed*} removes the suffix -> ./20070214_014700
  


• 
  ${prefix##*/} removes the prefix to the first / -> 20070214_014700
  


• 
  ${path##*processed/} also removes the prefix and leaves the filename -> AZ.AS..HHZ
  

The resulting target filename of the cp command is test/20070214_014700_AZ.AS..HHZ.

share|improve this answer

answered yesterday

FreddyFreddy

6,66011 gold badge66 silver badges2424 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Perfect, tested it out and it fits the bill exactly. Thanks a lot for the help, just starting to learn the command line so your example is very educational.
  
  – LividNoodle
  yesterday
  

  
  
  
  

add a comment | 

find . -type f -path "./*.a/processed/*" -exec sh -c '

for path; do

  prefix=${path%%.a/processed*}

  cp "$path" "test/${prefix##*/}_${path##*processed/}"

done

' sh {} +

Option -type f searches for regular files in the given path and the -exec option starts a shell script with find's result as arguments ({} +).
In the for-loop each argument is assigned to the path variable.

Example: If variable path is ./20070214_014700.a/processed/AZ.AS..HHZ, then

• 
  prefix=${path%%.a/processed*} removes the suffix -> ./20070214_014700
  


• 
  ${prefix##*/} removes the prefix to the first / -> 20070214_014700
  


• 
  ${path##*processed/} also removes the prefix and leaves the filename -> AZ.AS..HHZ
  

The resulting target filename of the cp command is test/20070214_014700_AZ.AS..HHZ.

share|improve this answer

answered yesterday

FreddyFreddy

6,66011 gold badge66 silver badges2424 bronze badges

find . -type f -path "./*.a/processed/*" -exec sh -c '

for path; do

  prefix=${path%%.a/processed*}

  cp "$path" "test/${prefix##*/}_${path##*processed/}"

done

' sh {} +

share|improve this answer

answered yesterday

FreddyFreddy

6,66011 gold badge66 silver badges2424 bronze badges

answered yesterday

FreddyFreddy

6,66011 gold badge66 silver badges2424 bronze badges

answered yesterday

FreddyFreddy

6,66011 gold badge66 silver badges2424 bronze badges